SSL — Self-supervised Learning

Fact

A training method where data creates its own labels for the model.

In Plain Words

Self-supervised learning is a worksheet with answers on the back. The AI guesses first, then flips to check.

It often gives a model its first big practice during pretraining. Later, it helps with language tasks and recognition.

Related Concepts

• Pretraining
• Masked Language Modeling
• Language Modeling
• Bidirectional Encoder Representations from Transformers

Pretraining
Self-supervised learning is often used in pretraining, so the model learns broad patterns first.

MLM
MLM hides part of the text and asks the model to guess it.

LM
LM predicts the next word, so the text provides the training signal.

BERT
BERT used self-supervised pretraining, then adapted to understanding tasks.