A regularization method that shrinks parameters to reduce overfitting.
Weight decay is a dog trainer for AI knobs. It says, “Easy, buddy. No need to bark at every leaf.”
During training, it gently pushes big weights down. This helps the model learn patterns, not memorize every weird example.
Regularization
Weight decay is a form of regularization used to control overfitting.
Parameter
Weight decay acts on parameters and nudges large weights smaller.
Adam
Weight decay is often set with Adam during training.
Lasso
Both limit parameters, but Lasso tends to make many weights zero.