Dataset · nvidia
Specialized pretraining data for Nemotron
nvidia/Nemotron-Pretraining-Specialized-v1
Updated corpus for Nemotron 3: math, code, English web, and synthetic data.
What's inside
Expansion of previously released Nemotron pretraining datasets: fresher, higher quality, and more diverse data across math, code, English Common Crawl, and large synthetic corpora. Created for the NVIDIA Nemotron 3 LLM family. The card mentions newly extracted code from Common Crawl and 2.5 trillion new English web tokens; metadata indicates a size of 10–100 million items.
What you can do with it
- Pretraining or fine-tuning LLMs in the Nemotron 3 style
- Adapting models on math and code
- Experiments with web and synthetic data mixtures
- Tasks
- Text generation
- Size
- 10M–100M rows
- Downloads this month
- 13.9K
- Likes
- 91
Models trained on it
- nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16Text generation 1.1M
- nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-NVFP4Text generation 641.7K
- nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-FP8Text generation 189.4K
- unsloth/Nemotron-3-Nano-30B-A3B-GGUFText generation 31.7K
- lmstudio-community/NVIDIA-Nemotron-3-Nano-30B-A3B-MLX-4bitText generation 30.5K
- lmstudio-community/NVIDIA-Nemotron-3-Nano-30B-A3B-MLX-8bitText generation 27.6K
- lmstudio-community/NVIDIA-Nemotron-3-Nano-30B-A3B-MLX-6bitText generation 27.2K
- lmstudio-community/NVIDIA-Nemotron-3-Nano-30B-A3B-MLX-5bitText generation 27K
How to load it
from datasets import load_dataset
ds = load_dataset("nvidia/Nemotron-Pretraining-Specialized-v1")