Figure 4.6 Illustration of the soft margin for a linear support vector machine (SVM).

(4.47)

Figure 2.7 of Chapter 2 is modified to reflect better the definitions introduced in this section and presented as Figure 4.6. Often, the optimization problem, Eq. (4.46), can be solved more easily in its dual formulation. The dual formulation provides also the key for extending SV machine to nonlinear functions.

      Lagrange function: The key idea here is to construct a Lagrange function from the objective function (primal objective function) and the corresponding constraints, by introducing a dual set of variables [95]. It can be shown that this function has a saddle point with respect to the primal and dual variables at the solution. So we have

(4.48)

Here, L is the Lagrangian, and η_i, α_i, are Lagrange multipliers. Hence, the dual variables in Eq. (4.48) have to satisfy positivity constraints, that is, where by , we jointly refer to α_i and In the saddle point, the partial derivatives of L with respect to the primal variables have to vanish; that is, , and = 0. Substituting these conditions into Eq. (4.48) yields the dual optimization problem:

(4.49)

In deriving Eq. (4.49), we already eliminated the dual variables η_i, through condition = 0, which can be reformulated as . Equation can be rewritten as , giving

(4.50)Скачать книгу