Dataset · CohereLabs
Multilingual Aya collection for NLP tasks
CohereLabs/aya_collection
A large multilingual collection of texts for classification, summarization, and translation.
What's inside
Aya Collection is a multilingual dataset containing texts and tabular data for text-classification, summarization, and translation tasks. It supports many languages including ace, afr, amh, ara, aze, ban, and others. Hugging Face hosts two identical downloads: this one is split by dataset names, while the Language Split version is split by language. Metadata indicates a size between 100M and 1B.
What you can do with it
- Fine-tuning a multilingual model for translation and summarization
- LoRA for low-resource languages
- Evaluating multilingual text classification
- Tasks
- Text classification, Summarization, Translation
- Languages
- Acehnese, Afrikaans, Amharic, Arabic, Azerbaijani, Balinese, Batak Toba, Belarusian and 4 more
- Size
- 100M–1B rows
- Downloads this month
- 11.9K
- Likes
- 252
Models trained on it
How to load it
from datasets import load_dataset
ds = load_dataset("CohereLabs/aya_collection")