Dataset · openbmb
UltraFeedback preference dataset
openbmb/UltraFeedback
64K prompts and 256K LLM responses with ratings for training reward and critic models.
What's inside
UltraFeedback is an English preference dataset. Prompts are gathered from UltraChat, ShareGPT, Evol-Instruct, TruthfulQA, FalseQA, and FLAN. For each prompt, several LLMs were queried to yield 4 responses each, totaling about 256K examples.
What you can do with it
- Train a reward model
- Run RLHF training
- Train a critic model to evaluate responses
- Tasks
- Text generation
- Languages
- English
- Size
- 10K–100K rows
- Downloads this month
- 17.8K
- Likes
- 449
Models trained on it
- legraphista/Gemma-2-9B-It-SPPO-Iter3-IMat-GGUFText generation 11.3K
- UCLA-AGI/Llama-3-Instruct-8B-SPPO-Iter3Text generation 9.4K
- UCLA-AGI/Llama-3-Instruct-8B-SPPO-Iter2Text generation 9.3K
- UCLA-AGI/Mistral7B-PairRM-SPPO-Iter3Text generation 8.9K
- UCLA-AGI/Mistral7B-PairRM-SPPO-Iter2Text generation 8.8K
- HuggingFaceH4/zephyr-7b-alphaText generation 6.2K
- bartowski/Gemma-2-9B-It-SPPO-Iter3-GGUFText generation 2.6K
- TheBloke/zephyr-7B-alpha-GGUF 2.6K
How to load it
from datasets import load_dataset
ds = load_dataset("openbmb/UltraFeedback")