A way to limit overfitting so a model works better on new data.
Regularization is like a strict coach at basketball practice. It stops the model from learning every weird bounce on one gym floor.
During training, it adds limits to the model. This helps the model do better on new data, not just the practice set.
Bias-Variance Tradeoff
Regularization lowers variance, but too much can make the model too simple.
ERM
Regularization adds extra limits on top of lowering training error.
Fine-tuning
Fine-tuning can overfit with little data, so regularization often helps.
Parameter
Many regularization methods limit the size of parameters.