Skip to content

Dataset · nvidia

Nemotron-CC v2.1 pretraining corpus

nvidia/Nemotron-CC-v2.1

Updated data for LLM pretraining: math, code, English Common Crawl, and synthetic data.

What's inside

The dataset expands previous Nemotron pretraining sets with fresher, higher-quality, and more diverse data. It includes math, code, English Common Crawl, and large synthetic corpora. The update adds newly mined code from Common Crawl and 2.5T new English web data; the set is designed for the Nemotron 3 family.

What you can do with it

  • Pretraining Nemotron family LLMs
  • Continuing pretraining on code and mathematics
  • Investigating the quality of web and synthetic data
Tasks
Text generation
Size
1B–10B rows
Downloads this month
35K
Likes
147
Access
on request on Hugging Face

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("nvidia/Nemotron-CC-v2.1")