Skip to content

Dataset · argilla

OpenHermes 2.5 alpha sample for DPO alignment

argilla/OpenHermes2.5-dpo-binarized-alpha

A test set of preference pairs based on OpenHermes 2.5 for LLM tuning via DPO without proprietary models.

What's inside

This is an early alpha version of the dataset (1,000 to 10,000 examples) collected by the Argilla team using the distilabel tool on top of the popular OpenHermes-2.5 set. Instead of closed OpenAI models for scoring and selecting the best answers, the open PairRM model was used here. The data is prepared in a format compatible with the ChatML standard.

What you can do with it

  • Testing language model alignment pipelines (DPO) in ChatML format
  • Experiments with the open preference evaluation stack (PairRM + distilabel)
  • Rapid debugging of SFT/DPO training on a small dialogue volume
Size
1K–10K rows
Downloads this month
105
Likes
64

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("argilla/OpenHermes2.5-dpo-binarized-alpha")