Skip to content

Dataset · HuggingFaceFW

FineTranslations-Edu: parallel texts in 500+ languages

HuggingFaceFW/finetranslations-edu

Educational subset from a massive corpus of translations between English and over 500 languages.

What's inside

This is the educational (Edu) version of the large-scale FineTranslations dataset from HuggingFaceFW, featuring only the top 10% best data evaluated by a special classifier. It contains between 100 million and 1 billion examples (part of the overall 1+ trillion token corpus) obtained by translating web texts from FineWeb2 into English.

What you can do with it

  • Training multilingual machine translation models, including low-resource languages
  • Pretraining and SFT of quality language models focused on educational content
Tasks
Text generation, Translation
Languages
Abkhazian, abq, abs, acm, adh, adi, Adyghe, Tunisian Arabic and 4 more
Size
100M–1B rows
Downloads this month
1.7K
Likes
34

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("HuggingFaceFW/finetranslations-edu")