Decoder-only Transformer

Fact

A Transformer design that predicts the next token, one step at a time.

In Plain Words

A decoder-only Transformer is like driving with only a rearview mirror. You see the road behind, but the next turn stays hidden until you get there.

You meet it in chatbots and code helpers. It is the main body plan behind many big language models.

Related Concepts

• Transformer architecture
• Autoregressive Model
• Generative Pre-trained Transformer
• Bidirectional Encoder Representations from Transformers

Transformer
A decoder-only Transformer is a core Transformer branch for making new text.

Autoregressive Model
It predicts the next token in order, so it can keep writing.

GPT
GPT made this design the main path for large language models.

BERT
BERT reads both sides of text, so it is more for reading tasks than for writing onward.