Dataset · facebook
Belebele Multilingual Benchmark
facebook/belebele
MRC benchmark with questions and 4-way multiple-choice options across 122 language varieties, including low-resource ones.
What's inside
Belebele is a multiple-choice reading comprehension benchmark. Questions are tied to short passages from FLORES-200 and feature four answer choices. The dataset covers 122 language varieties, ranging from high- to low-resource languages; questions are human-annotated.
What you can do with it
- Zero-shot evaluation of multilingual LLMs
- Comparison of mono- and multilingual models
- NLU testing on low-resource languages
- Tasks
- Question answering, Zero shot classification, Text classification, Multiple choice
- Languages
- Afrikaans, Amharic, Arabic, Azerbaijani, Assamese, Bambara, Bangla, Tibetan and 4 more
- Size
- 100K–1M rows
- Downloads this month
- 53.5K
- Likes
- 146
Models trained on it
- jondurbin/bagel-dpo-34b-v0.2Text generation 9.2K
- jondurbin/bagel-dpo-34b-v0.5Text generation 9.2K
- jondurbin/bagel-8b-v1.0Text generation 9.2K
- mradermacher/bagel-dpo-34b-v0.5-i1-GGUF 2.6K
- mradermacher/bagel-34b-v0.5-i1-GGUF 2.5K
- nlpai-lab/MixLIDToken classification 1.4K
- mradermacher/UNA-34Beagles-32K-bf16-v1-i1-GGUF 1.4K
- mradermacher/bagel-dpo-20b-v04-i1-GGUF 1.3K
How to load it
from datasets import load_dataset
ds = load_dataset("facebook/belebele")