Just a few days ago, I was talking with a friend of mine who works in the data science field as a consultant. My friend is doing well and is working with great companies, and I was wondering what kinds of sophisticated algorithms and methods they were employing to satisfy their needs.

I asked him to tell me what models and algorithms he most frequently employs for his jobs, and I was surprised by his answer. He told me that yes, he sometimes applies fancy models and algorithms, but the most part of his customers' needs are satisfied through regression models and quite basic classification algorithms. Are they working with unsophisticated customers? Not quite, but most of the time their customer wants to be provided with a model that gives them decisional instruments, and those kinds of models are able to play this role.

When we estimated our multiple linear regression model, we saw the meaning of its coefficients: the influence of a variation of a given x over the level of a response variable y. This is a decisional instrument, since it lets the business owner set the desired level of y, and on this basis derive the needed level of x, and even the right combination of x to obtain y.

On the other hand, giving the customer a black box model where they just know the final level of prediction without knowing how different predictors worked together and influenced the output, will actually be a lot less useful. This will hold true even if the black-box prediction is more accurate than the basic one. 

This exemplifies the trade-off between fitting and interpretability: models such as linear regression may not be the best in terms of fitting over a given dataset, but their estimation always produces an output that can be easily interpreted and can provide precious elements for planning and control activities. Nevertheless, they can't be considered as interpretable since they don't adapt to the shape of underlying trends of the phenomenon they are modelling. That is why they can be considered as an extreme: highly interpretable, poorly flexible.

On the other had, we find models that are highly flexible but poorly flexible. Those are models such as the support vector machine we are going to experiment with later on. Their greater quality is the ability to accurately model what is going on within the data they are provided, but if you ask them to help you in making a decision you would probably remain deluded.

So, what do you think: is it better to work with interpretable but rigid models, or the opposite?

There is no universal answer, and it is actually a matter of what the objectives of your analysis are:

• If your primary need is to obtain a model that helps you make decisions besides predicting the possible future outcome of a given phenomenon, you will go for models with a high level of interpretability, even sacrificing the overall accuracy of your predictions. Think for instance, about the expected value of GDP in a country as a function of macroeconomic variables. A policymaker will be interested to understand and weigh the relationship between the macroeconomic variables and the GDP, in order to prioritize their actions on those that result to be more relevant. They will be less interested in exactly predicting the future level of GDP for the next period (this will perhaps be of interest for some other purposes such as the definition of the public budget).
• If your primary need is to obtain an accurate prediction of a future event or perfectly model the phenomenon that is going on in your data, you will prefer models with a high level of flexibility. A good example of this situation could be a state of emergency where a hurricane is expected to come in the next days and you need to plan for the evacuation of an entire city. You are not actually interested in knowing what are the main variables that influenced the origination of the hurricane or its direction, what you actually care about is the day and possibly the hour when the hurricane will hit your city.