In this section, we will show the performance of our facial expression recognition system. In our test, we will keep the parameters of each learning algorithm the same and only change the feature extraction. We will evaluate the feature extraction with the number of clusters equaling 200, 500, 1,000, 1,500, 2,000, and 3,000.
The following table shows the accuracy of the system with the number of clusters equaling 200, 500, 1,000, 1,500, 2,000, and 3,000.
Table 1: The accuracy (%) of the system with 1,000 clusters
|
K = 1000 |
MLP |
SVM |
KNN |
Normal Bayes |
|---|---|---|---|---|
|
SIFT |
72.7273 |
93.1818 |
81.8182 |
88.6364 |
|
SURF |
61.3636 |
79.5455 |
72.7273 |
79.5455 |
|
BRISK |
61.3636 |
65.9091 |
59.0909 |
68.1818 |
|
KAZE |
50 |
79.5455 |
61.3636 |
77.2727 |
|
DAISY |
59.0909 |
77.2727 |
65.9091 |
81.8182 |
|
DENSE-SIFT |
20.4545 |
45.4545 |
43.1818 |
40.9091 |
Table 2: The accuracy (%) of the system with 500 clusters
|
K = 500 |
MLP |
SVM |
KNN |
Normal Bayes |
|---|---|---|---|---|
|
SIFT |
56.8182 |
70.4545 |
75 |
77.2727 |
|
SURF |
54.5455 |
63.6364 |
68.1818 |
79.5455 |
|
BRISK |
36.3636 |
59.0909 |
52.2727 |
52.2727 |
|
KAZE |
47.7273 |
56.8182 |
63.6364 |
65.9091 |
|
DAISY |
54.5455 |
75 |
63.6364 |
75 |
|
DENSE-SIFT |
27.2727 |
43.1818 |
38.6364 |
43.1818 |
Table 3: The accuracy (%) of the system with 200 clusters
|
K = 200 |
MLP |
SVM |
KNN |
Normal Bayes |
|---|---|---|---|---|
|
SIFT |
50 |
68.1818 |
65.9091 |
75 |
|
SURF |
43.1818 |
54.5455 |
52.2727 |
63.6364 |
|
BRISK |
29.5455 |
47.7273 |
50 |
54.5455 |
|
KAZE |
50 |
59.0909 |
72.7273 |
59.0909 |
|
DAISY |
45.4545 |
68.1818 |
65.9091 |
70.4545 |
|
DENSE-SIFT |
29.5455 |
43.1818 |
40.9091 |
31.8182 |
Table 4: The accuracy (%) of the system with 1,500 clusters
|
K = 1500 |
MLP |
SVM |
KNN |
Normal Bayes |
|---|---|---|---|---|
|
SIFT |
45.4545 |
84.0909 |
75 |
79.5455 |
|
SURF |
72.7273 |
88.6364 |
79.5455 |
86.3636 |
|
BRISK |
54.5455 |
72.7273 |
56.8182 |
68.1818 |
|
KAZE |
45.4545 |
79.5455 |
72.7273 |
77.2727 |
|
DAISY |
61.3636 |
88.6364 |
65.9091 |
81.8182 |
|
DENSE-SIFT |
34.0909 |
47.7273 |
38.6364 |
38.6364 |
Table 5: The accuracy (%) of the system with 2,000 clusters
|
K = 2000 |
MLP |
SVM |
KNN |
Normal Bayes |
|---|---|---|---|---|
|
SIFT |
63.6364 |
88.6364 |
81.8182 |
88.6364 |
|
SURF |
65.9091 |
84.0909 |
68.1818 |
81.8182 |
|
BRISK |
47.7273 |
68.1818 |
47.7273 |
61.3636 |
|
KAZE |
47.7273 |
77.2727 |
72.7273 |
75 |
|
DAISY |
77.2727 |
81.8182 |
72.7273 |
84.0909 |
|
DENSE-SIFT |
38.6364 |
45.4545 |
36.3636 |
43.1818 |
Table 6: The accuracy (%) of the system with 3,000 clusters
|
K = 3000 |
MLP |
SVM |
KNN |
Normal Bayes |
|---|---|---|---|---|
|
SIFT |
52.2727 |
88.6364 |
77.2727 |
86.3636 |
|
SURF |
59.0909 |
79.5455 |
65.9091 |
77.2727 |
|
BRISK |
52.2727 |
65.9091 |
43.1818 |
59.0909 |
|
KAZE |
61.3636 |
81.8182 |
70.4545 |
84.0909 |
|
DAISY |
72.7273 |
79.5455 |
70.4545 |
68.1818 |
|
DENSE-SIFT |
27.2727 |
47.7273 |
38.6364 |
45.4545 |
We can create graphs with the above results to compare the performance between features and learning algorithms in the following figure. We can see that SVM and Normal Bayes have better results than the others in most cases. The best result is 93.1818% for SVM and SIFT in 1,000 clusters. MLP has the lowest result in almost every case. One reason is that MLP requires lots of data to prevent over fitting. We only have around 160 training images. However, the feature size for each sample is between 100 and 150. Even with two hidden neurons, the number of weights is larger than the number of samples. KNN seems to work better than MLP but can't beat SVM and Normal Bayes.
Relationship between the performance of features and machine algorithms under different numbers of clusters
Effect of the number of centroids on the performance of features according to different machine algorithms
In the figure, Relationship between the performance of features and machine algorithms under different numbers of clusters, we have evaluated six features. SIFT gives the best results in most cases. DAISY is comparable to SIFT. In some cases, KAZE also gives good results. DENSE-SIFT is not a good choice for our facial expression problem since the results are poor. Moreover, the computation cost for DENSE features is really high. In conclusion, SIFT is still the most stable choice. However, SIFT is under patent. You may want to look at DAISY or KAZE. We recommend you do the evaluation on your data and choose the most suitable feature.
In the figure, Effect of the number of centroids on the performance of features according to different machine algorithms, we made a graph to visualize the effects of the number of clusters on performance. As you can see, the number of clusters differs between features. In SIFT, KAZE, and BRISK, the best number of clusters is 1,000. However, in SURF, DAISY, and DENSE-SIFT, 1,500 is a better choice. Basically, we don't want the number of clusters to be too large. The computation cost in kmeans increases with a larger number of clusters, especially in DENSE-SIFT.