Dataset · nvidia
Nemotron-CC v2.1 pretraining corpus
nvidia/Nemotron-CC-v2.1
Updated data for LLM pretraining: math, code, English Common Crawl, and synthetic data.
What's inside
The dataset expands previous Nemotron pretraining sets with fresher, higher-quality, and more diverse data. It includes math, code, English Common Crawl, and large synthetic corpora. The update adds newly mined code from Common Crawl and 2.5T new English web data; the set is designed for the Nemotron 3 family.
What you can do with it
- Pretraining Nemotron family LLMs
- Continuing pretraining on code and mathematics
- Investigating the quality of web and synthetic data
- Tasks
- Text generation
- Size
- 1B–10B rows
- Downloads this month
- 35K
- Likes
- 147
- Access
- on request on Hugging Face
Models trained on it
- nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16Text generation 1.1M
- nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-NVFP4Text generation 641.7K
- nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-FP8Text generation 189.4K
- skt/A.X-K2-NVFP4Text generation 84.2K
- unsloth/Nemotron-3-Nano-30B-A3B-GGUFText generation 31.7K
- lmstudio-community/NVIDIA-Nemotron-3-Nano-30B-A3B-MLX-4bitText generation 30.5K
- lmstudio-community/NVIDIA-Nemotron-3-Nano-30B-A3B-MLX-8bitText generation 27.6K
- lmstudio-community/NVIDIA-Nemotron-3-Nano-30B-A3B-MLX-6bitText generation 27.2K
How to load it
from datasets import load_dataset
ds = load_dataset("nvidia/Nemotron-CC-v2.1")