Dataset · nvidia
Nemotron code and math pre-training corpus
nvidia/Nemotron-Pretraining-Code-v2
NVIDIA's updated pre-training corpus with code, math, English web text, and synthetics.
What's inside
Nemotron-Pre-Training-Dataset-v2.1 expands NVIDIA's previous pre-training sets with higher quality and more diverse data on mathematics, code, English Common Crawl, and large synthetic corpora. The card mentions new code extraction from Common Crawl and 2.5T new English web data. The dataset was created for the NVIDIA Nemotron 3 model family; volume ranges from 100 million to 1 billion records.
What you can do with it
- Pre-train or continue pre-training code-LLMs
- Fine-tune the model on math and code
- Evaluate code generation quality on the NVIDIA corpus
- Tasks
- Text generation
- Size
- 100M–1B rows
- Downloads this month
- 5.5K
- Likes
- 139
- Access
- on request on Hugging Face
Models trained on it
- nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16Text generation 1.1M
- nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-NVFP4Text generation 641.7K
- nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-FP8Text generation 189.4K
- skt/A.X-K2-NVFP4Text generation 84.2K
- unsloth/Nemotron-3-Nano-30B-A3B-GGUFText generation 31.7K
- lmstudio-community/NVIDIA-Nemotron-3-Nano-30B-A3B-MLX-4bitText generation 30.5K
- lmstudio-community/NVIDIA-Nemotron-3-Nano-30B-A3B-MLX-8bitText generation 27.6K
- lmstudio-community/NVIDIA-Nemotron-3-Nano-30B-A3B-MLX-6bitText generation 27.2K
How to load it
from datasets import load_dataset
ds = load_dataset("nvidia/Nemotron-Pretraining-Code-v2")