Dataset · edinburgh-dawg
Relabeled MMLU benchmark (5,700 questions)
edinburgh-dawg/mmlu-redux-2.0
A manually cleaned set of 5,700 questions across 57 English disciplines for precise LLM evaluation.
What's inside
The dataset is a subset of 5,700 questions from the classic MMLU benchmark across 57 subjects, which have been re-verified and manually relabeled. The work was published at NAACL 2025, and the authors continue to make targeted corrections to found errors in math and algebra tasks.
What you can do with it
- Reliable evaluation of language model knowledge and logic without the noise of the original MMLU
- Compare checkpoints when fine-tuning models on academic tasks
- Tasks
- Question answering
- Languages
- English
- Size
- 1K–10K rows
- Downloads this month
- 63.9K
- Likes
- 38
Models trained on it
How to load it
from datasets import load_dataset
ds = load_dataset("edinburgh-dawg/mmlu-redux-2.0")