Dataset · nvidia
Math and code for LLM pre-training
nvidia/Nemotron-Pretraining-Code-v1
Pre-training corpus for generative models featuring math, code, and multilingual Q&A.
What's inside
A pre-training dataset for generative models that preserves valuable mathematical and code data, supplemented by diverse multilingual question-answer pairs. It supports the NVIDIA Nemotron Nano 2 family, including 9B models.
What you can do with it
- Pre-training Nemotron Nano 2
- Fine-tuning code and math LLMs
- SFT on multilingual Q&A
- Tasks
- Text generation
- Size
- 100M–1B rows
- Downloads this month
- 3K
- Likes
- 87
- Access
- on request on Hugging Face
Models trained on it
- nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16Text generation 1.1M
- nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-NVFP4Text generation 641.7K
- nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-FP8Text generation 189.4K
- unsloth/Nemotron-3-Nano-30B-A3B-GGUFText generation 31.7K
- lmstudio-community/NVIDIA-Nemotron-3-Nano-30B-A3B-MLX-4bitText generation 30.5K
- lmstudio-community/NVIDIA-Nemotron-3-Nano-30B-A3B-MLX-8bitText generation 27.6K
- lmstudio-community/NVIDIA-Nemotron-3-Nano-30B-A3B-MLX-6bitText generation 27.2K
- lmstudio-community/NVIDIA-Nemotron-3-Nano-30B-A3B-MLX-5bitText generation 27K
How to load it
from datasets import load_dataset
ds = load_dataset("nvidia/Nemotron-Pretraining-Code-v1")