RMSProp — Root Mean Square Propagation

Fact

An optimizer that adjusts each learning step using recent squared gradients.

In Plain Words

RMSProp is like a bike with smart gears. On a steep hill, it shifts down and stops the wobble.

It trains neural networks. Each weight gets its own step size, so steep parts bounce less.

Related Concepts

• Stochastic Gradient Descent
• Adaptive Gradient Algorithm
• Adaptive Moment Estimation optimizer

SGD
RMSProp builds on SGD and gives each weight its own step size.

AdaGrad
RMSProp uses a moving average, so AdaGrad’s steps shrink less over time.

Adam
Adam combines RMSProp’s adaptive step size with momentum.