Skip to content

Dataset · HuggingFaceH4

Preference pairs for chat models

HuggingFaceH4/ultrafeedback_binarized

Processed UltraFeedback: 64k prompts with model answers and GPT-4 scores aggregated into preference pairs.

What's inside

This is a preprocessed version of UltraFeedback used to train Zephyr-7B-beta. Originally, the set contains 64k prompts, each with four answers from open and proprietary models; GPT-4 evaluated them according to specified criteria. The binarized version is presented in tabular and text format in English.

What you can do with it

  • Train reward model for RLHF
  • Fine-tune chat model via DPO
  • Check generation quality based on preferences
Tasks
Text generation
Languages
English
Size
100K–1M rows
Downloads this month
35K
Likes
363

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("HuggingFaceH4/ultrafeedback_binarized")