Dataset · CohereLabs
Multilingual MMLU in 42 Languages
CohereLabs/Global-MMLU
MMLU questions in 42 languages with machine translation, professional editing, and cultural sensitivity markup.
What's inside
The dataset combines MMLU questions in 42 languages, including English. Some translations are machine-generated, while others are professionally done and supplemented with crowdsourced post-editing. For a subset of 2,850 questions per language, cultural sensitivity annotations are available: Culturally Sensitive or Culturally Agnostic. The format is text.
What you can do with it
- Evaluating multilingual LLMs on MMLU
- Researching benchmark cultural sensitivity
- Testing models in 42 languages
- Languages
- English, Arabic, Bangla, Spanish, French, Hindi, Russian, German and 4 more
- Size
- 100K–1M rows
- Downloads this month
- 55.6K
- Likes
- 169
Models trained on it
- mradermacher/Atomight-V2.2-0.5B-UltraThink-GGUF 572
- mradermacher/Atomight-V2.2-UltraThink-0.5B-GGUF 365
- usermma/Atomight-V2.2-0.5B-UltraThink-GGUF-mradermacher 232
- NovatasticRoScript/Atomight-V2.2-UltraThink-0.5BText generation 30
- onnx-community/Atomight-V2.2-UltraThink-0.5B-ONNXText generation 16
- randomlysjfsgshzbzf/MultipleModelsMultimodal 0
How to load it
from datasets import load_dataset
ds = load_dataset("CohereLabs/Global-MMLU")