Dataset · openai
Multi-needle long context benchmark
openai/mrcr
A dataset for testing LLMs on finding and distinguishing multiple needles in a long context.
What's inside
OpenAI MRCR is a long-context benchmark with multiple needles in a haystack. The model must distinguish multiple hidden context fragments; the task extends the complexity of Gemini's MRCR approach. Data is open for reproducibility; size ranges from 1 to 10 thousand examples, modalities are tables and text.
What you can do with it
- Evaluating LLMs on long context
- Testing multi-needle retrieval
- Comparing models using the MRCR benchmark
- Size
- 1K–10K rows
- Downloads this month
- 28.9K
- Likes
- 235
Models trained on it
How to load it
from datasets import load_dataset
ds = load_dataset("openai/mrcr")