Now let's apply PCA to a face-recognition problem. Face recognition is the supervised classification task of identifying a person from an image of his or her face. In this example, we will use a data set called Our Database of Faces from AT&T Laboratories, Cambridge. The data set contains ten images each of forty people. The images were created under different lighting conditions, and the subjects varied their facial expressions. The images are gray scale and 92 x 112 pixels in dimension. The following is an example image:

While these images are small, a feature vector that encodes the intensity of every pixel will have 10,304 dimensions. Training from such high-dimensional data could require many samples to avoid over-fitting. Instead, we will use PCA to compactly represent the images in terms of a small number of principal components.

We can reshape the matrix of pixel intensities for an image into a vector, and create a matrix of these vectors for all of the training images. Each image is a linear combination of this data set's principal components. In the context of face recognition, these principal components are called eigenfaces. The eigenfaces can be thought of as standardized components of faces. Each face in the data set can be expressed as some combination of the eigenfaces, and can be approximated as a combination of the most important eigenfaces:

>>> from os import walk, path
>>> import numpy as np
>>> import mahotas as mh
>>> from sklearn.cross_validation import train_test_split
>>> from sklearn.cross_validation import cross_val_score
>>> from sklearn.preprocessing import scale
>>> from sklearn.decomposition import PCA
>>> from sklearn.linear_model import LogisticRegression
>>> from sklearn.metrics import classification_report
>>> X = []
>>> y = []

We begin by loading the images into NumPy arrays, and reshaping their matrices into vectors:

>>> for dir_path, dir_names, file_names in walk('data/att-faces/orl_faces'):
>>>     for fn in file_names:
>>>         if fn[-3:] == 'pgm':
>>>             image_filename = path.join(dir_path, fn)
>>>             X.append(scale(mh.imread(image_filename, as_grey=True).reshape(10304).astype('float32')))
>>>             y.append(dir_path)
>>> X = np.array(X)

We then randomly split the images into training and test sets, and fit the PCA object on the training set:

>>> X_train, X_test, y_train, y_test = train_test_split(X, y)
>>> pca = PCA(n_components=150)

We reduce all of the instances to 150 dimensions and train a logistic regression classifier. The data set contains forty classes; scikit-learn automatically creates binary classifiers using the one versus all strategy behind the scenes:

>>> X_train_reduced = pca.fit_transform(X_train)
>>> X_test_reduced = pca.transform(X_test)
>>> print 'The original dimensions of the training data were', X_train.shape
>>> print 'The reduced dimensions of the training data are', X_train_reduced.shape
>>> classifier = LogisticRegression()
>>> accuracies = cross_val_score(classifier, X_train_reduced, y_train)

Finally, we evaluate the performance of the classifier using cross-validation and a test set. The average per-class F1 score of the classifier trained on the full data was 0.94, but required significantly more time to train and could be prohibitively slow in an application with more training instances:

>>> print 'Cross validation accuracy:', np.mean(accuracies), accuracies
>>> classifier.fit(X_train_reduced, y_train)
>>> predictions = classifier.predict(X_test_reduced)
>>> print classification_report(y_test, predictions)

The following is the output of the script:

The original dimensions of the training data were (300, 10304)
The reduced dimensions of the training data are (300, 150)
Cross validation accuracy: 0.833841819347 [ 0.82882883  0.83        0.84269663]
             precision    recall  f1-score   support

data/att-faces/orl_faces/s1       1.00      1.00      1.00         2
data/att-faces/orl_faces/s10       1.00      1.00      1.00         2
data/att-faces/orl_faces/s11       1.00      0.60      0.75         5
...
data/att-faces/orl_faces/s9       1.00      1.00      1.00         2

avg / total       0.92      0.89      0.89       100