Japan Womenfs University Journal (Faculty of Integrated Arts and Social Sciences), Vol. 17 (2006), 59-71
A Justification of Rotation in Principal Component
Analysis:
Projective viewpoint of PCA
YASUHARU OKAMOTO
Abstract
Rotation in principal component analysis (PCA) is justified from projective point of view. PCA is interpreted as orthogonal projection of data, and components are constructed as coordinates of the projected data with respect to some basis. From the viewpoint of projection, any bases for the space of the projected data are equally valid, but a particular one may have substantive meaning in the research area. Transition from the current basis to another one induces change in coordinates, which is represented by transformation matrix, orthogonal or oblique.
Optimal orthogonal projection is obtained from the singular value decomposition (SVD) of the data matrix and results in the solution numerically equivalent to usual PCA. But, from the viewpoint of projection, transformation of components is interpreted as transformation of coordinates induced by change of bases, which can be understood geometrically with clear intuition. The principal components in usual PCA are coordinates of the projected data for the basis composed of right singular vectors of the data matrix.
Full-text (pdf-file): Click here.