Distillation — Model Distillation

Fact

A training method for moving big-model skills into a smaller model.

In Plain Words

Distillation is like squeezing a huge cookbook onto a fridge magnet. You lose the tiny tips, but dinner starts faster.

It makes smaller AI models for phones and laptops. You trade a little skill for speed.

Related Concepts

• Foundation Model
• Fine-tuning
• Local Large Language Model

Foundation-model
Distillation squeezes a Foundation-model's skills into a smaller model.

Fine-tuning
Distillation can keep special skills after Fine-tuning.

Local-LLM
Distillation makes a Local-LLM need less local hardware.