Dataset · Helsinki-NLP
Multilingual translation of the Nemotron-CC corpus into 36 languages
Helsinki-NLP/nemotron-cc-translated
A massive 2.4 trillion token parallel corpus translated from English into 36 languages.
What's inside
A large-scale text corpus from Helsinki-NLP obtained by machine translating the high-quality portion of the English Nemotron-CC dataset using OPUS-MT and HPLT-MT models. Version 1.0 contains over 156 million documents (over 70 billion source English tokens) aligned across 36 languages, including Bosnian, Bulgarian, Catalan, Czech, Danish, and German. The total volume is over 2.4 trillion tokens.
What you can do with it
- Training machine translation systems across 36 language directions
- Pretraining multilingual large language models (LLMs)
- Tasks
- Translation, Text generation
- Languages
- Bosnian, Bulgarian, Catalan, Czech, Danish, German, Greek, English and 4 more
- Size
- 1B–10B rows
- Downloads this month
- 150.2K
- Likes
- 14
How to load it
from datasets import load_dataset
ds = load_dataset("Helsinki-NLP/nemotron-cc-translated")