Attention — Attention Mechanism

Fact

A mechanism for focusing on the most relevant parts during information processing.

In Plain Words

Attention is the model’s highlighter pen. It marks the juicy bits, not every boring comma.

It helps the model pick key clues in your prompt. You meet it inside Transformers and LLMs.

Related Concepts

• Transformer architecture
• Context Window
• Large Language Model

Transformer
Attention helps a Transformer decide which information matters most.

Context-window
A longer Context-window gives Attention more clues to sort.

LLM
Attention helps an LLM connect far-apart text.