Skip to content

Dataset · nvidia

NVIDIA Common Crawl code corpus for pre-training

nvidia/Nemotron-CC-Code-v1

Updated code text dataset with 100 million to 1 billion examples for pre-training language models.

What's inside

The dataset is part of the Nemotron-Pre-Training-Dataset-v2.1 collection and contains 100 million to 1 billion text examples. The data was obtained through a new extraction of programming code from Common Crawl web archives specifically for training the NVIDIA Nemotron 3 model family.

What you can do with it

  • Pre-training large language models (LLMs) with strong programming skills
  • Fine-tuning existing text models to improve code generation and understanding
Tasks
Text generation
Size
100M–1B rows
Downloads this month
31K
Likes
38
Access
on request on Hugging Face

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("nvidia/Nemotron-CC-Code-v1")