Skip to content

Dataset · Anthropic

Preference data for RLHF

Anthropic/hh-rlhf

Response pairs with human preferences for helpfulness and safety to train reward models.

What's inside

The repository contains two types of data: human preferences on helpfulness and harmlessness from work on training a helpful and harmless assistant with RLHF. This data is intended for training preference/reward models for subsequent RLHF, rather than for supervised training of conversational agents. Size is 100K–1M, text modality; language is not specified in metadata.

What you can do with it

  • Training reward models
  • RLHF tuning of the assistant
  • Evaluating preferences for helpfulness and safety
Size
100K–1M rows
Downloads this month
47K
Likes
2.5K

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("Anthropic/hh-rlhf")