Bellman Equation

Fact

A formula that links today’s value to rewards now and later.

In Plain Words

Bellman Equation is like planning a road trip with snacks. A stop is great if it has donuts now and cookies later.

It helps AI score a move by adding now and later. You meet it in RL, Q-Learning, and DP.

Related Concepts

• Reinforcement Learning
• Markov Decision Process
• Q-Learning
• Dynamic Programming

RL
The Bellman Equation is the core formula for long-term reward in RL.

MDP
The Bellman Equation is usually built on MDP state changes.

Q-Learning
Q-Learning updates can be seen as sample guesses at the Bellman Equation.

DP
DP uses the Bellman Equation to compute values or better policies.