Dataset · HuggingFaceH4
Preference pairs for chat models
HuggingFaceH4/ultrafeedback_binarized
Processed UltraFeedback: 64k prompts with model answers and GPT-4 scores aggregated into preference pairs.
What's inside
This is a preprocessed version of UltraFeedback used to train Zephyr-7B-beta. Originally, the set contains 64k prompts, each with four answers from open and proprietary models; GPT-4 evaluated them according to specified criteria. The binarized version is presented in tabular and text format in English.
What you can do with it
- Train reward model for RLHF
- Fine-tune chat model via DPO
- Check generation quality based on preferences
- Tasks
- Text generation
- Languages
- English
- Size
- 100K–1M rows
- Downloads this month
- 35K
- Likes
- 363
Models trained on it
- TinyLlama/TinyLlama-1.1B-Chat-v1.0Text generation 961K
- HuggingFaceH4/zephyr-7b-betaText generation 49.9K
- stabilityai/stablelm-zephyr-3bText generation 24.1K
- tenyx/Llama3-TenyxChat-70BText generation 9.3K
- Zyphra/Zamba2-1.2B-instructText generation 7.6K
- TheBloke/zephyr-7B-beta-GGUF 4.5K
- sapienzanlp/Minerva-7B-instruct-v1.0Text generation 2K
- TheBloke/stablelm-zephyr-3b-GGUF 1.9K
How to load it
from datasets import load_dataset
ds = load_dataset("HuggingFaceH4/ultrafeedback_binarized")