Skip to content

Dataset · PKU-Alignment

Preferences and Q-A for LLM safety

PKU-Alignment/PKU-SafeRLHF

Data for training and evaluating safe responses: preferences, question-answer pairs, and English prompts.

What's inside

The dataset contains data for language model safety research: single-dimension preferences, question-answer pairs, and prompts; may include offensive or harmful content. Language is English, size is from 100k to 1 million records, format is tabular and text data. Author is PKU-Alignment.

What you can do with it

  • Train a reward model for RLHF
  • Fine-tune LLMs for safe responses via SFT
  • Test harmful content filters
Tasks
Text generation
Languages
English
Size
100K–1M rows
Downloads this month
14.9K
Likes
207

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("PKU-Alignment/PKU-SafeRLHF")