A supervised rule for updating weights to shrink squared errors.
It is like a bowling coach with a tiny steering wheel. A small miss gets a small turn. A huge miss gets more turning.
It fixes weights after each mistake in early neural nets and adaptive filters. Its tiny steps helped inspire Gradient Descent training.
Supervised Learning
The rule uses the right answer to find the error, then updates weights.
Gradient Descent
The rule moves weights downhill on squared error.
Perceptron
The rule helped early linear neurons learn from each mistake.
SGD
Its one-example-at-a-time tiny updates pointed toward later SGD.