Elastic Net regression is a classification algorithm that overcomes the limitations of the lasso(least absolute shrinkage and selection operator) method which uses a penalty function in its L1 regularization. Elastic Net regression is a hybrid approach that blends both penalizations of the L2 and L1 regularization of lasso and ridge methods.
It finds an estimator in a two-stage procedure i.e first for each fixed λ2 it finds the ridge regression coefficients and then does a lasso regression type shrinkage which does a double amount of shrinkage which eventually leads to increased bias and poor predictions. Rescaling the coefficients of the naive version of the elastic net by multiplying the estimated coefficients by (1 + λ2) is done to improve the prediction performance. Elastic Net regression is used in:
- Metric learning
- Portfolio optimization
- Cancer prognosis
Elastic Net regression always aims at minimizing the following loss function:
Elastic Net also allows us to tune the alpha parameter where alpha = 0 corresponds to Ridge regression and alpha = 1 to Lasso regression. Similarly, when alpha = 0, the penalty function reduces to the L1(ridge) regularization, and when alpha = 1, the penalty function reduces to L2(lasso) regularization. Therefore, we can choose an alpha value between 0 and 1 to optimize the Elastic Net and this will shrink some coefficients and set some to 0 for sparse selection. In Elastic Net regression, the lambda hyper-parameter is mostly and heavily dependent on the alpha hyper-parameter. Now let’s implement elastic net regression in R programming.
Implementation in R
mtcars(motor trend car road test) comprises fuel consumption, performance and 10 aspects of automobile design for 32 automobiles. It comes pre-installed with dplyr package in R.
Performing Elastic Net Regression on Dataset
Using the Elastic Net regression algorithm on the dataset by training the model using features or variables in the dataset.
- Training of Elastic Net Regression model:
The Elastic Net regression model is trained to find the optimum alpha and lambda values.
- Model elastic_model:
The Elastic Net regression model uses the alpha value as 0.6242021 and lambda value as 1.801398. RMSE was used to select the optimal model using the smallest value.
- Model Prediction:
The model is predicted using the Y dataset and values are shown.
- Multiple R-Squared:
The multiple R-Squared values of disp is 0.9514679.
The mixing percentage is plotted with RMSE scores with different values of the regularization parameter.
- Poisson Regression in R Programming
- Logistic Regression in R Programming
- Regression Analysis in R Programming
- Perform Linear Regression Analysis in R Programming - lm() Function
- Polynomial Regression in R Programming
- Random Forest Approach for Regression in R Programming
- Lasso Regression in R Programming
- Regression and its Types in R Programming
- Regression using k-Nearest Neighbors in R Programming
- Decision Tree for Regression in R Programming
- R-squared Regression Analysis in R Programming
- Ridge Regression in R Programming
- Quantile Regression in R Programming
- Regression with Categorical Variables in R Programming
- Getting the Modulus of the Determinant of a Matrix in R Programming - determinant() Function
- Set or View the Graphics Palette in R Programming - palette() Function
- tidyr Package in R Programming
- Get Exclusive Elements between Two Objects in R Programming - setdiff() Function
- Intersection of Two Objects in R Programming - intersect() Function
- Add Leading Zeros to the Elements of a Vector in R Programming - Using paste0() and sprintf() Function
So, Elastic Net regression applications are used in many sectors of industry and with full capacity.
If you like GeeksforGeeks and would like to contribute, you can also write an article using contribute.geeksforgeeks.org or mail your article to email@example.com. See your article appearing on the GeeksforGeeks main page and help other Geeks.
Please Improve this article if you find anything incorrect by clicking on the "Improve Article" button below.