KL Divergence — Kullback-Leibler Divergence

Fact

A one-way score for how different two probability distributions are.

In Plain Words

You were 99% sure the vending machine had cookies. It drops kale chips. KL Divergence is the betrayal score.

It compares one set of chances with another. People use it to keep a model from drifting too far.

Related Concepts

• Cross-Entropy Loss
• Reinforcement Learning from Human Feedback
• Model Distillation

Cross-entropy Loss
Cross-entropy is closely tied to KL divergence.

RLHF
RLHF often uses KL to keep policy updates from moving too far.

Distillation
Distillation uses KL to measure how close the student is to the teacher.