Dataset · CohereLabs
Aya multilingual instructions
CohereLabs/aya_dataset
204K prompt-response pairs across 65 languages for training and evaluating multilingual LLMs.
What's inside
Collected by the Aya Open Science Initiative community via an annotation platform. It contains 204K human-written prompt-completion pairs plus annotator demographics. It covers 65 languages (71 including dialects), with a scale of 100K to 1M examples.
What you can do with it
- Fine-tune a multilingual LLM on human instructions
- Evaluate response quality in low-resource languages
- Create LoRA adapters for multi-language instructions
- Tasks
- Other
- Languages
- Amharic, Arabic, Moroccan Arabic, Najdi Arabic, acq, Egyptian Arabic, apc, Bangla and 4 more
- Size
- 100K–1M rows
- Downloads this month
- 26.5K
- Likes
- 382
Models trained on it
- Horizon-Labs/prompt-injection-guard-baseText classification 7.3K
- Horizon-Labs/prompt-injection-guard-smallText classification 5.4K
- CohereLabs/aya-101 3K
- BSC-LT/ALIA-40b-instruct-2606-GGUFText generation 2.9K
- mradermacher/Aitana-7B-S-Instruct-i1-GGUFText generation 2.7K
- BSC-LT/salamandra-7b-instruct-2606Text generation 1.8K
- yunusemrejr/BlazingKiwi-EN-135MText generation 1.7K
- mradermacher/salamandra-7b-instruct-2606-i1-GGUF 1.6K
How to load it
from datasets import load_dataset
ds = load_dataset("CohereLabs/aya_dataset")