Recurrent Transformer

Fact

A Transformer design that carries notes from one text chunk to the next.

In Plain Words

It is like reading a long book with sticky notes. The next chapter gets the important notes.

It helps AI follow longer text. It is used for reading and writing language.

Related Concepts

• Transformer architecture
• Recurrent Neural Network
• Context Window

Transformer
It adds shared notes between text chunks to a Transformer.

RNN
It borrows the RNN idea of passing earlier information forward.

Context-window
It keeps older context beyond one fixed window.