Dataset · argilla
OpenHermesPreferences preference pairs
argilla/OpenHermesPreferences
About 1 million response pairs with preference scores for reward models and LLM alignment.
What's inside
An English text dataset containing approximately 1 million AI preferences. Built from teknium/OpenHermes-2.5, where original responses are compared against generations from Mixtral-8x7B-Instruct-v0.1 and Nous-Hermes-2-Yi-34B, with PairRM ranking them. Suitable for training preference models and aligning language models.
What you can do with it
- Train a reward model on preference pairs.
- Perform RLHF alignment of a language model.
- Evaluate a model based on response ranking.
- Tasks
- Text generation
- Languages
- English
- Size
- 100K–1M rows
- Downloads this month
- 1.2K
- Likes
- 224
Models trained on it
- Zyphra/Zamba2-1.2B-instructText generation 7.6K
- mradermacher/Llama-3.2-SUN-2.5B-chat-i1-GGUF 2.3K
- mradermacher/Llama-3.2-SUN-HDIC-1B-Instruct-i1-GGUF 1.4K
- mradermacher/Llama-3.2-SUN-1B-chat-i1-GGUF 1.2K
- Zyphra/Zamba2-2.7B-instructText generation 1K
- meditsolutions/Llama-3.2-SUN-2.5B-chatText generation 933
- mradermacher/Llama-3.2-SUN-2.5B-chat-GGUF 669
- mradermacher/ogno-monarch-jaskier-merge-7b-OH-PREF-DPO-v2-i1-GGUF 542
How to load it
from datasets import load_dataset
ds = load_dataset("argilla/OpenHermesPreferences")