Dataset · opencompass
NeedleBench benchmark for long text fact retrieval
opencompass/NeedleBench
A test suite from OpenCompass with 1 to 10 thousand text tasks to evaluate LLM retrieval and reasoning over large documents.
What's inside
The dataset is part of the OpenCompass project and contains between 1 and 10 thousand text examples for question-answering tasks. It includes a series of test scenarios: single-needle retrieval, multi-needle retrieval, and multi-hop reasoning based on several fragments. Specific language is not specified in the metadata, but the description is provided in English.
What you can do with it
- Verifying single-fact extraction accuracy from long context (Needle-in-a-Haystack)
- Evaluating LLM capability to synthesize information from multiple parts of a large document
- Comparing long-context models within the OpenCompass ecosystem
- Tasks
- Question answering
- Size
- 1K–10K rows
- Downloads this month
- 6.8K
- Likes
- 7
How to load it
from datasets import load_dataset
ds = load_dataset("opencompass/NeedleBench")