Dataset · argilla
OpenHermes 2.5 alpha sample for DPO alignment
argilla/OpenHermes2.5-dpo-binarized-alpha
A test set of preference pairs based on OpenHermes 2.5 for LLM tuning via DPO without proprietary models.
What's inside
This is an early alpha version of the dataset (1,000 to 10,000 examples) collected by the Argilla team using the distilabel tool on top of the popular OpenHermes-2.5 set. Instead of closed OpenAI models for scoring and selecting the best answers, the open PairRM model was used here. The data is prepared in a format compatible with the ChatML standard.
What you can do with it
- Testing language model alignment pipelines (DPO) in ChatML format
- Experiments with the open preference evaluation stack (PairRM + distilabel)
- Rapid debugging of SFT/DPO training on a small dialogue volume
- Size
- 1K–10K rows
- Downloads this month
- 105
- Likes
- 64
Models trained on it
- kalisai/Nusantara-2.7b-Indo-ChatText generation 843
- mradermacher/AlphaMonarch-laser-i1-GGUF 831
- mradermacher/dpo-binarized-NeuralTrix-7B-i1-GGUF 816
- frankenmerger/MiniLlama-1.8b-Chat-v0.1Text generation 714
- mradermacher/NeuralReyna-Mini-1.8B-v0.3-i1-GGUF 650
- mradermacher/dpo-binarized-NeuralTrix-7B-GGUF 502
- mradermacher/AlphaMonarch-dora-GGUF 385
- mradermacher/dpo-binarized-NeutrixOmnibe-7B-i1-GGUF 293
How to load it
from datasets import load_dataset
ds = load_dataset("argilla/OpenHermes2.5-dpo-binarized-alpha")