Dataset · Anthropic
Preference data for RLHF
Anthropic/hh-rlhf
Response pairs with human preferences for helpfulness and safety to train reward models.
What's inside
The repository contains two types of data: human preferences on helpfulness and harmlessness from work on training a helpful and harmless assistant with RLHF. This data is intended for training preference/reward models for subsequent RLHF, rather than for supervised training of conversational agents. Size is 100K–1M, text modality; language is not specified in metadata.
What you can do with it
- Training reward models
- RLHF tuning of the assistant
- Evaluating preferences for helpfulness and safety
- Size
- 100K–1M rows
- Downloads this month
- 47K
- Likes
- 2.5K
Models trained on it
- StrandsAgents/strands-decider-2B-qwen3.5-v1-2610Text classification 25K
- OpenAssistant/reward-model-deberta-v3-large-v2Text classification 10.5K
- garak-llm/attackgeneration-toxicity_gpt2 8.4K
- sileod/deberta-v3-base-tasksource-nliZero shot classification 8K
- mradermacher/archangel_sft-kto_llama30b-i1-GGUF 3.1K
- mradermacher/mpt-7b-chat-i1-GGUF 2.2K
- llm-blender/PairRM-hfText generation 2.2K
- Ray2333/gpt2-large-helpful-reward_modelText classification 2K
How to load it
from datasets import load_dataset
ds = load_dataset("Anthropic/hh-rlhf")