Skip to content

Dataset · edinburgh-dawg

Relabeled MMLU benchmark (5,700 questions)

edinburgh-dawg/mmlu-redux-2.0

A manually cleaned set of 5,700 questions across 57 English disciplines for precise LLM evaluation.

What's inside

The dataset is a subset of 5,700 questions from the classic MMLU benchmark across 57 subjects, which have been re-verified and manually relabeled. The work was published at NAACL 2025, and the authors continue to make targeted corrections to found errors in math and algebra tasks.

What you can do with it

  • Reliable evaluation of language model knowledge and logic without the noise of the original MMLU
  • Compare checkpoints when fine-tuning models on academic tasks
Tasks
Question answering
Languages
English
Size
1K–10K rows
Downloads this month
63.9K
Likes
38

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("edinburgh-dawg/mmlu-redux-2.0")