A test of whether a model can find one fact in a long input.
Needle-haystack is Where’s Waldo for AI, but the page is a phone book. One tiny sentence hides in a giant stack.
People use it on long documents and code. It shows if the model forgets facts as the text gets longer.
Context-window
Needle-haystack tests if the model can find facts in a long context.
RAG
Needle-haystack tests finding a needle in the input, not outside lookup.
Benchmark contamination
If test samples leak, the score may look good but not be trustworthy.