At the beginning of the chapter on regression techniques, we said that regression analysis can be conducted for dual purposes; one of these is to understand and weigh the effects of the independent variable on the dependent variable. In other words, with this method, we can select predictors that have a greater influence on the response of the model. Stepwise regression is a method of selecting independent variables in order to choose a set of predictors that have the best relationship with the dependent variable. Among variable selection algorithms, we have three methods:

• The forward method starts with an empty model in which no predictors are selected; in the first step, the variable with the most significant association at the statistical level is added. At each subsequent step, the remaining variable with the largest statistically significant association is added into the model. This process continues until it is no longer variable with a statistically significant association with the dependent variable.
• The backward method begins with a model that includes all the variables. Then, we proceed step by step to delete the variables starting from the one with the least significant association.
• The stepwise method moves back and forth between the two processes by adding and removing variables that gain or lose significance in the various model adjustments (with the addition or reinsertion of a variable).

In the following figure, the criteria adopted from these three methods to select the variables are shown:

To learn how to use a feature selection algorithm, we perform a stepwise regression analysis step by step.