One of the most relevant objects when talking about the performance of classification models is the confusion matrix which, despite its name, is a crucial instrument to get a clear view of how your model is performing. Moreover, it is employed to derive other useful performance metrics.

This matrix is actually a table with two columns and two rows, like the following one:

Let's build up a bit of story here to better understand this matrix. We are trying to predict if a customer will buy a newly introduced car model or not, based on some behavioral and descriptive attributes we have. For instance, we could collect a dataset showing for each customer if he bought that specific model or not, together with a series of attributes such as the annual income and number of sons . Let's say we introduced our model six months ago and now we want to design a new marketing campaign to target the cluster of customers that showed the greatest interest in our new model during these six months.

We then try some kind of model which, based on the values assumed from the mentioned explanatory variables, predicts if the given customer will buy our new model or not. What the confusion matrix does is simply compare the observed type of car bought with the predicted one.

For instance, the observed TRUE-predicted TRUE cell tells us that for 54 cases, both the model and the reality showed the new model being bought by the customer. Similarly, the observed FALSE-predicted X shows the number of cases where both the model and the observed data showed our customer not buying the new car model. In the middle, you find a mixed case, that is, a prediction error, where either the model predicted TRUE and the observed was FALSE or the model predicted FALSE and the observed was TRUE. 

It turns our that the confusion matrix is considered as a really relevant object for data mining purposes because it can be employed to compute at least three useful metrics:

• Accuracy
• Sensitivity
• Specificity