Skip to content

Dataset · nvidia

Math corpus from Common Crawl

nvidia/Nemotron-CC-Math-v1

133 billion math and code tokens from Common Crawl for LLM pre-training.

What's inside

Nemotron-CC-Math is a large, high-quality math corpus extracted from Common Crawl. According to the card, its volume is 133 billion tokens; the data was used in Nemotron pre-training and preserves valuable math and code information. Metadata lists the size as 100M-1B, while the paper description states 133 billion tokens. The modality is text.

What you can do with it

  • Pre-training math LLMs
  • Fine-tuning models to solve math and code
  • Evaluating the quality of math pre-training data
Tasks
Text generation
Size
100M–1B rows
Downloads this month
13.8K
Likes
105
Access
on request on Hugging Face

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("nvidia/Nemotron-CC-Math-v1")