Regularization
Ridge Regression
- L2 regularization or squared
- We add penalty term along with SSR to optimize
- Rigde regression does not make slope of any variable to zero
- If a variable is not very important then it’s slope will be closer to zero and its parameter will be shrinked
- Ridge regression is useful when all the variables in a model are useful
Lasso Regression
- L1 regularization or absolute
- We add absolute value of the slope to the SSR to optimize
- The value of slope can become zero
- We can use lasso where unimportant variables are included in the model
Use both Lasso and Ridge regression to get best of the both worlds
- Elastic Net combines both types of regularization