BERT — Bidirectional Encoder Representations from Transformers

Fact

A two-way pre-trained language model built on the Transformer encoder.

In Plain Words

BERT is the careful kid with a worksheet. It checks both sides of the blank before answering.

It is good at understanding text. You meet it in search, text sorting, and fact pulling.

Related Concepts

• Transformer architecture
• Masked Language Modeling
• Vector Embedding
• Generative Pre-trained Transformer

Transformer
BERT is built on the Transformer encoder.

MLM
BERT trains by hiding some words and guessing them.

Embedding
BERT can make meaning vectors for words or sentences.

GPT
BERT focuses more on text understanding than generation.