Skip to content

Dataset · edinburgh-dawg

Cleaned subset of the MMLU benchmark (3,000 questions)

edinburgh-dawg/mmlu-redux

Manually re-verified questions across 30 subjects from MMLU in English for accurate model evaluation.

What's inside

The set includes 3,000 text questions in English selected from 30 disciplines of the MMLU benchmark and manually re-annotated to fix errors. The work was accepted at NAACL 2025, with the card noting the existence of a newer MMLU-Redux-2.0 version with all 57 subjects.

What you can do with it

  • More accurate evaluation of general knowledge and reasoning in English LLMs without the noise of original MMLU
  • Model validation in scientific and educational tasks across 30 disciplines
Tasks
Question answering
Languages
English
Size
1K–10K rows
Downloads this month
3.9K
Likes
38

How to load it

from datasets import load_dataset

ds = load_dataset("edinburgh-dawg/mmlu-redux")