Dataset · math-ai
AutoMathText-2.5 math and STEM corpus with 2T tokens
math-ai/AutoMathText-2.5
Cleaned bilingual (EN/ZH) corpus exceeding 2 trillion tokens on mathematics, code, and reasoning for LLM training.
What's inside
The dataset contains over 2 trillion tokens (from 10B to 100B documents) in English and Chinese, covering web content, mathematics, code, and reasoning chains. The data was gathered from over 50 sources, undergoing three-tier deduplication and AI-based quality filtering.
What you can do with it
- Pre-training and fine-tuning (continued pretraining) language models with an emphasis on mathematics, code, and logic
- Creating bilingual (EN/ZH) STEM models
- Tasks
- Text generation, Question answering
- Languages
- English, Chinese
- Size
- 1B–10B rows
- Downloads this month
- 39.4K
- Likes
- 3
How to load it
from datasets import load_dataset
ds = load_dataset("math-ai/AutoMathText-2.5")