Figure 4.12 The general application of PCA.

For a single‐channel EEG the Karhunen–Loéve transform is used to decompose the ith channel signal into a set of weighted orthogonal basis functions:

      (4.117)

      where Φ = {ϕ_k } is the set of orthogonal basis functions. The weights w_{i, k} are then calculated as:

      (4.118)

Often noise is added to the signal, i.e. x_i (n) = s_i (n) + v_i (n), where v_i (n) is additive noise. This degrades the decorrelation process. The weights are then estimated in order to minimize a function of the error between the signal and its expansion by the orthogonal basis, i.e. e _i = x_i − Φw_i . Minimization of the error in this case is generally carried out by solving the least‐squares problem. In a typical application of PCA as depicted in Figure 4.12, the signal and noise subspaces are separated by means of some classification procedure.

      4.9.1 Singular Value Decomposition

      Singular value decomposition (SVD) is often used for solving the least‐squares (LS) problem. This is performed by decomposition of the M × M square autocorrelation matrix R into its eigenvalue matrix Λ = diag(λ₁, λ₂, … λ _M ) and an M × M orthogonal matrix of eigenvectors V, i.e. R = VΛV^H , where (.) ^H denotes Hermitian (conjugate transpose) operation. Moreover, if A is an M × M data matrix such that R = A^H A then there exist an M × M orthogonal matrix U, an M × M orthogonal matrix V, and an M × M diagonal matrix ∑ with diagonal elements equal to , such that:

      (4.119)

      Hence ∑ ² = Λ. The columns of U are called left singular vectors and the rows of V^H are called right singular vectors. If A is rectangular N × M matrix of rank k then U will be N × N and ∑ will be:

      (4.120)

      where S = diag(σ₁, σ₂, … σ _k ), where σ _i = . For such a matrix the Moore–Penrose pseudo‐inverse is defined as an M × N matrix A^† defined as:

      (4.121)

where ∑ ^† is an M × N matrix defined as:

      (4.122)

      A ^† has a major role in the solutions of least‐squares problems, and S ⁻¹ is a k × k diagonal matrix with elements equal to the reciprocals of the singular values of A, i.e.

      (4.123)

      In order to see the application of the SVD in solving the LS problem consider the error vector e defined as:

(4.124)

where d is the desired signal vector and Ah is the estimate . To find h we replace A with its SVD in the above equation (Eq. 4.124) and find h, which thereby minimizes the squared Euclidean norm of the error vector, ‖e ²‖. By using the SVD we obtain:

      (4.125)

      or equivalently

      (4.126)

      Since U is a unitary matrix, ‖e ²‖ = ‖U^H e ‖². Hence, the vector h that minimizes ‖e ²‖ also minimizes ‖U^H e ‖². Finally, the unique solution as an optimum h (coefficient vector) may be expressed as [43]:

      (4.127)

      where k is the rank of A. Alternatively, as the optimum least‐squares coefficient vector:

      (4.128)

      Performing PCA is equivalent to performing an SVD on the covariance matrix. PCA uses the same concept as SVD and orthogonalization to decompose the data into its constituent uncorrelated orthogonal components such that the autocorrelation matrix is diagonalized. Each eigenvector represents a principal component and the individual eigenvalues are numerically related to the variance they capture in the direction of the principal components.