ELMo — Embeddings from Language Models

Fact

A language model giving each word a meaning from its sentence.

In Plain Words

ELMo is like hearing “bat” at school. Gym class says baseball. Science class says tiny vampire.

It reads nearby words before it builds a word vector. You meet it in sentiment tools and question-answer bots.

Related Concepts

• Vector Embedding
• Long Short-Term Memory
• Language Modeling
• Bidirectional Encoder Representations from Transformers

Embedding
ELMo makes an Embedding change when the sentence changes.

LSTM
The first ELMo used LSTMs to read words from both directions.

LM
ELMo learned language first, then used that skill to make word meanings.

BERT
BERT took context pretraining further.