At some point, after we have removed redundant features and dropped irrelevant ones, we will often still find that we have too many features. No matter what learning method we use, they all perform badly and, given the huge feature space, we understand that they actually cannot do better. We have to get rid of features, even though common sense tells us that they are valuable. Another situation where we need to reduce the feature dimension, and where feature selection does not help much, is when we want to visualize data. Then, we need to have, at most, three dimensions at the end to provide any meaningful graphs.

Enter feature projection methods. They restructure the feature space to make it more accessible to the model, or simply cut down the dimensions to two or three so that we can show dependencies visually.

Again, we can distinguish feature projection methods as being linear or nonlinear. Also, as seen before in the Selecting features section, we will present one method for each type (principal component analysis as a linear and nonlinear version of multidimensional scaling). Although they are widely known and used, they are only a selection of the more interesting and powerful feature projection methods available.