Skip to content

Dataset · Helsinki-NLP

Multilingual translation of the Nemotron-CC corpus into 36 languages

Helsinki-NLP/nemotron-cc-translated

A massive 2.4 trillion token parallel corpus translated from English into 36 languages.

What's inside

A large-scale text corpus from Helsinki-NLP obtained by machine translating the high-quality portion of the English Nemotron-CC dataset using OPUS-MT and HPLT-MT models. Version 1.0 contains over 156 million documents (over 70 billion source English tokens) aligned across 36 languages, including Bosnian, Bulgarian, Catalan, Czech, Danish, and German. The total volume is over 2.4 trillion tokens.

What you can do with it

  • Training machine translation systems across 36 language directions
  • Pretraining multilingual large language models (LLMs)
Tasks
Translation, Text generation
Languages
Bosnian, Bulgarian, Catalan, Czech, Danish, German, Greek, English and 4 more
Size
1B–10B rows
Downloads this month
150.2K
Likes
14

How to load it

from datasets import load_dataset

ds = load_dataset("Helsinki-NLP/nemotron-cc-translated")