Dataset · HuggingFaceFW
FineTranslations-Edu: parallel texts in 500+ languages
HuggingFaceFW/finetranslations-edu
Educational subset from a massive corpus of translations between English and over 500 languages.
What's inside
This is the educational (Edu) version of the large-scale FineTranslations dataset from HuggingFaceFW, featuring only the top 10% best data evaluated by a special classifier. It contains between 100 million and 1 billion examples (part of the overall 1+ trillion token corpus) obtained by translating web texts from FineWeb2 into English.
What you can do with it
- Training multilingual machine translation models, including low-resource languages
- Pretraining and SFT of quality language models focused on educational content
- Tasks
- Text generation, Translation
- Languages
- Abkhazian, abq, abs, acm, adh, adi, Adyghe, Tunisian Arabic and 4 more
- Size
- 100M–1B rows
- Downloads this month
- 1.7K
- Likes
- 34
Models trained on it
How to load it
from datasets import load_dataset
ds = load_dataset("HuggingFaceFW/finetranslations-edu")