Dataset · PKU-Alignment
Preferences and Q-A for LLM safety
PKU-Alignment/PKU-SafeRLHF
Data for training and evaluating safe responses: preferences, question-answer pairs, and English prompts.
What's inside
The dataset contains data for language model safety research: single-dimension preferences, question-answer pairs, and prompts; may include offensive or harmful content. Language is English, size is from 100k to 1 million records, format is tabular and text data. Author is PKU-Alignment.
What you can do with it
- Train a reward model for RLHF
- Fine-tune LLMs for safe responses via SFT
- Test harmful content filters
- Tasks
- Text generation
- Languages
- English
- Size
- 100K–1M rows
- Downloads this month
- 14.9K
- Likes
- 207
Models trained on it
- PKU-Alignment/beaver-7b-v1.0-costReinforcement learning 2.2K
- mradermacher/piano-medley-7b-i1-GGUF 1.8K
- mradermacher/piano-medley-7b-GGUF 790
- PKU-Alignment/beaver-7b-v1.0-rewardReinforcement learning 648
- mradermacher/AmberSafe-i1-GGUF 574
- mradermacher/Llama-3-OffsetBias-8B-GGUF 467
- mradermacher/beaver-7b-v2.0-GGUFReinforcement learning 397
- mradermacher/beaver-7b-v1.0-GGUFReinforcement learning 328
How to load it
from datasets import load_dataset
ds = load_dataset("PKU-Alignment/PKU-SafeRLHF")