A model's test mistakes can drop, rise, then drop again as it gets bigger.
Double Descent is like upgrading a toy car. One button bumps the couch. A few buttons crash harder. Full controls finally drive straight.
It shows test mistakes can fall, rise, then fall again as a model gets bigger. In large model tests, the old U-shaped curve can fool you.
Overparameterization
Double Descent often appears after a model has more parameters than examples.
Bias-Variance Tradeoff
Double Descent challenges the old U-shaped view of test mistakes.
Scaling-law
Double Descent shows bigger models can sometimes still get better.
SLT
Double Descent pushes SLT to explain generalization again.