Skip to content

Dataset · a-m-team

Large distilled reasoning corpus for LLMs

a-m-team/AM-DeepSeek-Distilled-40M

A large-scale text dataset with difficulty grading for SFT and enhancing model reasoning skills.

What's inside

The repository features an open subset of a large dataset (10M to 100M records) created during the DeepDistill study to improve LLM logical capabilities. The data in Chinese and English is sorted by difficulty levels and presented in text and tabular formats. According to the authors, training a 72B model using only SFT on the full corpus yielded 79.2 points on the AIME 2024 benchmark.

What you can do with it

  • SFT fine-tuning of language models for complex logic and math tasks
  • Distilling reasoning skills into more compact models
  • Experiments with training on difficulty-graded data
Tasks
Text generation
Languages
Chinese, English
Size
10M–100M rows
Downloads this month
3K
Likes
56

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("a-m-team/AM-DeepSeek-Distilled-40M")