Bahdanau Attention

Fact

An attention method for lining up generated words with input positions.

In Plain Words

Bahdanau Attention is like a kid doing homework with one finger on the textbook. For each answer, the finger scoots to the right line.

You meet it in older translation models. As each word is written, it looks back at the most useful input word.

Related Concepts

• Attention Mechanism
• Sequence-to-Sequence Model
• Self-Attention
• Transformer architecture

Attention
Bahdanau Attention is an early classic form of Attention.

Seq2Seq
It gives Seq2Seq a moving link between input and output.

Self-Attention
Self-Attention extends this focus idea inside the sequence.

Transformer
Transformer keeps the attention idea and makes it much bigger.