LLM-as-a-judge

Fact

A way to use an LLM to score answers automatically.

In Plain Words

It is like putting AI on a talent-show panel. It gives scores fast, but may still boo the best singer.

People use it to grade model answers and tests. It saves time, but humans should check important scores.

Related Concepts

• Large Language Model
• Benchmark contamination
• Algorithmic Bias
• Human-in-the-loop

LLM
LLM-as-a-judge uses an LLM as an automatic scorer or comparer.

Benchmark contamination
Benchmark contamination can make LLM-as-a-judge scores look better than they are.

AI-bias
AI-bias in the judge model can make the scores unfair.

Human-in-the-loop
Human-in-the-loop helps catch bad scores in important cases.