C4 — Colossal Clean Crawled Corpus

Fact

A cleaned English web-text dataset made from Common Crawl for pretraining AI models.

In Plain Words

C4 is like taking the internet’s messy school binder. You toss the gum wrappers, then give AI the readable pages.

Models read C4 during pretraining. It helps them learn basic English patterns from cleaner web pages.

Related Concepts

• Common Crawl
• Pretraining
• Text-to-Text Transfer Transformer

Common Crawl
C4 starts from Common Crawl pages and cleans them into training text.

Pretraining
C4 is often used as a large reading source for language model pretraining.

T5
The T5 paper introduced C4 and made it a classic dataset.