Skip to content

Dataset · openbmb

UltraFeedback preference dataset

openbmb/UltraFeedback

64K prompts and 256K LLM responses with ratings for training reward and critic models.

What's inside

UltraFeedback is an English preference dataset. Prompts are gathered from UltraChat, ShareGPT, Evol-Instruct, TruthfulQA, FalseQA, and FLAN. For each prompt, several LLMs were queried to yield 4 responses each, totaling about 256K examples.

What you can do with it

  • Train a reward model
  • Run RLHF training
  • Train a critic model to evaluate responses
Tasks
Text generation
Languages
English
Size
10K–100K rows
Downloads this month
17.8K
Likes
449

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("openbmb/UltraFeedback")