Skip to content

Dataset · nvidia

Math and code for LLM pre-training

nvidia/Nemotron-Pretraining-Code-v1

Pre-training corpus for generative models featuring math, code, and multilingual Q&A.

What's inside

A pre-training dataset for generative models that preserves valuable mathematical and code data, supplemented by diverse multilingual question-answer pairs. It supports the NVIDIA Nemotron Nano 2 family, including 9B models.

What you can do with it

  • Pre-training Nemotron Nano 2
  • Fine-tuning code and math LLMs
  • SFT on multilingual Q&A
Tasks
Text generation
Size
100M–1B rows
Downloads this month
3K
Likes
87
Access
on request on Hugging Face

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("nvidia/Nemotron-Pretraining-Code-v1")