Skip to content

Dataset · argilla

OpenHermesPreferences preference pairs

argilla/OpenHermesPreferences

About 1 million response pairs with preference scores for reward models and LLM alignment.

What's inside

An English text dataset containing approximately 1 million AI preferences. Built from teknium/OpenHermes-2.5, where original responses are compared against generations from Mixtral-8x7B-Instruct-v0.1 and Nous-Hermes-2-Yi-34B, with PairRM ranking them. Suitable for training preference models and aligning language models.

What you can do with it

  • Train a reward model on preference pairs.
  • Perform RLHF alignment of a language model.
  • Evaluate a model based on response ranking.
Tasks
Text generation
Languages
English
Size
100K–1M rows
Downloads this month
1.2K
Likes
224

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("argilla/OpenHermesPreferences")