Regularization

Ridge Regression

  • L2 regularization or squared
  • We add penalty term along with SSR to optimize
  • Rigde regression does not make slope of any variable to zero
  • If a variable is not very important then it’s slope will be closer to zero and its parameter will be shrinked
  • Ridge regression is useful when all the variables in a model are useful

Lasso Regression

  • L1 regularization or absolute
  • We add absolute value of the slope to the SSR to optimize
  • The value of slope can become zero
  • We can use lasso where unimportant variables are included in the model

Use both Lasso and Ridge regression to get best of the both worlds

  • Elastic Net combines both types of regularization