Trying to overcome the issue of no perfect hyperplane, we place our blade somewhere within the tank, trying to place it in a way that allows us to put most of the yellow balls on one side and most of the red balls on the opposite side.

Moving from all to most is what the support vector machine is about, but let's first see what these support vectors are. You have now placed your blade somewhere. I see it, and I can tell you that there is a better position for the blade, and I invite you to find it out by placing it differently. What would you do?

After thinking about it carefully, you discover that what makes the difference in the performance of your classifier are not the balls that are far from the plane, which we can consider as safely classified from nearly every position of the blade but are rather the more next to the plane itself. You would actually discover that the most relevant ones are the points lying on the margin we have described before. Those points are the so-called support vectors, and they are the most relevant for the algorithmic procedure that produces the final selection of the best possible plane.

But, what is now the best possible plane? It is the one that maximizes the number of yellow balls on the yellow side, or alternatively the one that minimizes the number of red balls on the yellow side.

A final step to finally arrive to support vector machines is to remove the idea of the hyperplane being linear. Support vector machines are a group of algorithms defined to find out the best possible hyperplane, defined with different functional forms, from the simple linear to the non-linear and even radial one.