The power of elastic net is that, it performs the feature extraction that ridge regression does not and it will group the features that LASSO fails to do. Again, LASSO will tend to select one feature from a group of correlated ones and ignore the rest. Elastic net does this by including a mixing parameter, alpha, in conjunction with lambda. Alpha will be between 0 and 1 and as before, lambda will regulate the size of the penalty. Please note that an alpha of zero is equal to ridge regression and an alpha of one is equivalent to LASSO. Essentially, we are blending the L1 and L2 penalties by including a second tuning parameter with a quadratic (squared) term of the beta coefficients. We will end up with the goal of minimizing (RSS + λ[(1-alpha) (sum|Bj|²)/2 + alpha (sum |Bj|)])/N).

Let's put these techniques to test. We will primarily utilize the leaps, glmnet, and caret packages to select the appropriate features and thus the appropriate model in our business case.