A benchmark that tests LLMs across many different tasks.
BIG-bench is like a school talent show for AI. One minute it solves a puzzle. Next, it trips over common sense.
People use it to compare LLMs across many skills. A high score helps, but it is not an IQ badge.
LLM
BIG-bench tests the broad skill limits of an LLM.
MMLU
MMLU is the standard exam. BIG-bench is the weird talent show.
Emergence
BIG-bench can show sudden jumps in model skill.
Benchmark contamination
Scores can rise if BIG-bench questions enter training data.