Dataset · edinburgh-dawg
Cleaned subset of the MMLU benchmark (3,000 questions)
edinburgh-dawg/mmlu-redux
Manually re-verified questions across 30 subjects from MMLU in English for accurate model evaluation.
What's inside
The set includes 3,000 text questions in English selected from 30 disciplines of the MMLU benchmark and manually re-annotated to fix errors. The work was accepted at NAACL 2025, with the card noting the existence of a newer MMLU-Redux-2.0 version with all 57 subjects.
What you can do with it
- More accurate evaluation of general knowledge and reasoning in English LLMs without the noise of original MMLU
- Model validation in scientific and educational tasks across 30 disciplines
- Tasks
- Question answering
- Languages
- English
- Size
- 1K–10K rows
- Downloads this month
- 3.9K
- Likes
- 38
How to load it
from datasets import load_dataset
ds = load_dataset("edinburgh-dawg/mmlu-redux")