Skip to content

Dataset · math-ai

AutoMathText-2.5 math and STEM corpus with 2T tokens

math-ai/AutoMathText-2.5

Cleaned bilingual (EN/ZH) corpus exceeding 2 trillion tokens on mathematics, code, and reasoning for LLM training.

What's inside

The dataset contains over 2 trillion tokens (from 10B to 100B documents) in English and Chinese, covering web content, mathematics, code, and reasoning chains. The data was gathered from over 50 sources, undergoing three-tier deduplication and AI-based quality filtering.

What you can do with it

  • Pre-training and fine-tuning (continued pretraining) language models with an emphasis on mathematics, code, and logic
  • Creating bilingual (EN/ZH) STEM models
Tasks
Text generation, Question answering
Languages
English, Chinese
Size
1B–10B rows
Downloads this month
39.4K
Likes
3

How to load it

from datasets import load_dataset

ds = load_dataset("math-ai/AutoMathText-2.5")