Avoiding Overfitting and Colinearity with Regularization
Fighting 2 very important problems in quant finance
2 of the most common problems we face when building trading models are overfitting and colinearity. Regularization is a technique that we can use to combat both of those problems.
More broadly, regularization is a process that converts the answer of a problem to a simpler one.
This can be done in multiple different ways. Some explicit regularization techniques are penalties and constraints. Implicit regularization techniques are early stopping, robust loss functions, discarding outliers etc.
Bias-variance tradeoff tells us that as we increase regularization to learn broader patterns in our data our variance decreases but therefore our bias (inaccuracy) increases.
Table of Content
Norm Penalization and Constraints
Lasso, Ridge and Elastic Net Regression
Polynomial Regression
Model Selection
Robust Covariance Matrix
Final Remarks