Skip to content

Dataset · davidheineman

Dataset for SFT and RL training

davidheineman/tiny-rl-refined-tasks-v5

Text tasks for SFT and RL with prediction tokens and model rewards.

What's inside

The dataset contains English text data ranging in size from 1M to 10M. Each task includes tokens for SFT prediction and RL sampling, as well as scores from the Skywork 3B reward model.

What you can do with it

  • Training and fine-tuning language models using SFT
  • Conducting RLHF experiments
Tasks
Text generation
Languages
English
Size
1M–10M rows
Downloads this month
39.4K
Likes
0

How to load it

from datasets import load_dataset

ds = load_dataset("davidheineman/tiny-rl-refined-tasks-v5")