GAE — Generalized Advantage Estimation

Fact

A low-noise method for estimating how much an action helped a policy.

In Plain Words

GAE is like grading a taco run, not one missed turn. Check the whole ride before you judge.

It estimates how much each action helped, with less noise. In Policy Gradient and robot training, it makes updates less jumpy.

Related Concepts

• Policy Gradient
• Actor-Critic
• Temporal-Difference Learning
• Proximal Policy Optimization

Policy Gradient
GAE gives Policy Gradient a steadier advantage estimate.

Actor-Critic
The critic estimates value, and GAE makes the advantage signal steadier.

TD Learning
GAE uses multi-step TD errors to balance bias and variance.

PPO
PPO often uses GAE to calculate each action's advantage.