A training method for learning language by guessing hidden words.
MLM is like a worksheet with sticky notes over some words. The AI reads around each blank and fills in the missing word.
It is common in pretraining. It helps models get good at understanding text.
BERT
BERT became known for using MLM to understand context.
Pretraining
MLM is often used in pretraining with huge piles of text.
SSL
MLM makes questions from the text itself, so it is SSL.
LM
MLM is a type of LM, but it does not only guess the next word.