RoPE — Rotary Positional Embedding

Fact

A position method that uses rotating vectors inside attention.

In Plain Words

RoPE is like giving each word a tiny clock face. Each word points to a different hour, so distance is easy to spot.

It works inside attention layers. It helps Transformers keep word order steadier in long text.

Related Concepts

• Positional Encoding
• Self-Attention
• Transformer architecture
• Context Window

Positional Encoding
RoPE is a kind of Positional Encoding. It writes order as angles.

Self-Attention
RoPE helps Self-Attention notice the distance between words.

Transformer
Many Transformers use RoPE instead of older position methods.

Context-window
RoPE can help models handle positions beyond shorter text.