Dataset · CohereLabs
Multilingual benchmark of examination tests
CohereLabs/include-base-44
22,637 four-choice questions across 44 languages to evaluate LLM knowledge and logic.
What's inside
The dataset contains 22,637 text multiple-choice questions (MCQs) in 44 languages, including Albanian, Arabic, Armenian, Azerbaijani, Belarusian, and Bengali. The tasks are extracted from real academic and professional exams and cover 57 different topics.
What you can do with it
- Evaluate knowledge and logical capabilities of multilingual LLMs
- Test models on low-resource languages
- Compare response quality of language models in a real linguistic environment
- Tasks
- Multiple choice
- Languages
- Albanian, Arabic, Armenian, Azerbaijani, Belarusian, Bangla, Basque, Bulgarian and 4 more
- Size
- 10K–100K rows
- Downloads this month
- 14.4K
- Likes
- 53
How to load it
from datasets import load_dataset
ds = load_dataset("CohereLabs/include-base-44")