A Transformer design with compressed memory for very long context.
Infini-attention is a Transformer with a tiny desk and a magic junk drawer. The desk holds today’s page. The drawer keeps squished notes from the whole book.
It helps AI read huge text without keeping every bit in full. You will mostly see it in long-context AI research.