Dataset · a-m-team
Large distilled reasoning corpus for LLMs
a-m-team/AM-DeepSeek-Distilled-40M
A large-scale text dataset with difficulty grading for SFT and enhancing model reasoning skills.
What's inside
The repository features an open subset of a large dataset (10M to 100M records) created during the DeepDistill study to improve LLM logical capabilities. The data in Chinese and English is sorted by difficulty levels and presented in text and tabular formats. According to the authors, training a 72B model using only SFT on the full corpus yielded 79.2 points on the AIME 2024 benchmark.
What you can do with it
- SFT fine-tuning of language models for complex logic and math tasks
- Distilling reasoning skills into more compact models
- Experiments with training on difficulty-graded data
- Tasks
- Text generation
- Languages
- Chinese, English
- Size
- 10M–100M rows
- Downloads this month
- 3K
- Likes
- 56
Models trained on it
How to load it
from datasets import load_dataset
ds = load_dataset("a-m-team/AM-DeepSeek-Distilled-40M")