Skip to content

Dataset · openai

Multi-needle long context benchmark

openai/mrcr

A dataset for testing LLMs on finding and distinguishing multiple needles in a long context.

What's inside

OpenAI MRCR is a long-context benchmark with multiple needles in a haystack. The model must distinguish multiple hidden context fragments; the task extends the complexity of Gemini's MRCR approach. Data is open for reproducibility; size ranges from 1 to 10 thousand examples, modalities are tables and text.

What you can do with it

  • Evaluating LLMs on long context
  • Testing multi-needle retrieval
  • Comparing models using the MRCR benchmark
Size
1K–10K rows
Downloads this month
28.9K
Likes
235

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("openai/mrcr")