Skip to content

Dataset · nvidia

Code data for Nemotron 3 pre-training

nvidia/Nemotron-Pretraining-Code-v3

NVIDIA code corpus metadata for pre-training Nemotron 3 family LLMs.

What's inside

Part of the Nemotron Pretraining Data collection designed to improve programming skills of the Nemotron 3 family LLMs. Contains metadata corresponding to source code updates in Nemotron-Pretraining-Code-v1 and v2 versions. Language is code, metadata size is 100M-1B, task is text-generation.

What you can do with it

  • Pre-train or fine-tune code-LLMs
  • Verify code generation of a Nemotron-like model
  • Assemble a pretraining mixture for code
Tasks
Text generation
Languages
Code
Size
100M–1B rows
Downloads this month
1.5K
Likes
77

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("nvidia/Nemotron-Pretraining-Code-v3")