Dataset · HuggingFaceH4
OpenOrca-based preference pairs for DPO tuning
HuggingFaceH4/orca_dpo_pairs
Preprocessed English dataset for language model alignment using DPO.
What's inside
Contains 10k to 100k English text examples. This is a preference-training-ready version of the OpenOrca dataset, which is based on extended FLAN data from the Orca paper. The set has already been used to create strong model checkpoints with preference tuning.
What you can do with it
- Language model alignment via DPO
- Fine-tuning chat models to follow instructions and reason
- Training reward models to evaluate response quality
- Tasks
- Text classification, Token classification, Table question answering, Question answering, Zero shot classification, Summarization
- Languages
- English
- Size
- 10K–100K rows
- Downloads this month
- 6.7K
- Likes
- 32
Models trained on it
- mradermacher/starchat2-15b-v0.1-i1-GGUF 376
- mradermacher/starchat2-15b-v0.1-GGUF 246
- bartowski/starchat2-15b-v0.1-GGUFText generation 170
- HuggingFaceH4/starchat2-15b-v0.1Text generation 159
- amd/Zebra-Llama-3B-14MLA-14Mamba-DPO 123
- tensorblock/starchat2-15b-v0.1-GGUF 60
- skymizer/Llama2-7b-sft-chat-custom-template-dpoText generation 51
- amd/X-EcoMLA-1B8B-fixed-kv64-DPO 32
How to load it
from datasets import load_dataset
ds = load_dataset("HuggingFaceH4/orca_dpo_pairs")