Dataset · openai
MMMLU: MMLU in 14 languages
openai/MMMLU
MMLU test questions across 57 topics translated into 14 languages by professional translators.
What's inside
MMLU is a general knowledge benchmark across 57 categories: from basic level to law, physics, history, and computer science. The test set was translated by professional translators into 14 languages; metadata specifies ar, bn, de, es, fr, hi, and others. Format is question-answer pairs.
What you can do with it
- Evaluate multilingual LLMs on general knowledge
- Compare benchmark translation quality
- Test QA systems on exam-level questions
- Tasks
- Question answering
- Languages
- Arabic, Bangla, German, Spanish, French, Hindi, Indonesian, Italian and 4 more
- Size
- 100K–1M rows
- Downloads this month
- 15.4K
- Likes
- 539
Models trained on it
- GOBA-AI-Labs/PrunedHub-Qwen3-30B-A3B-EN-80pct-MxMoEText generation 290
- TOk-Atsuru/gpt-oss-20b-pruned-28xText generation 222
- Marc432/vozmarcText → speech 85
- GOBA-AI-Labs/PrunedHub-Qwen3.5-35B-A3B-80pctText generation 78
- cs-552-2026-nlpowerpuffs/multilingual_model 59
- GOBA-AI-Labs/PrunedHub-Qwen3-30B-A3B-JP-80pct-MxMoEText generation 49
- rithesh10/ytmodelQuestion answering 2
- Bossmarc747/innovatehub-ai-modelMultimodal 0
How to load it
from datasets import load_dataset
ds = load_dataset("openai/MMMLU")