Skip to content

Dataset · HuggingFaceH4

OpenOrca-based preference pairs for DPO tuning

HuggingFaceH4/orca_dpo_pairs

Preprocessed English dataset for language model alignment using DPO.

What's inside

Contains 10k to 100k English text examples. This is a preference-training-ready version of the OpenOrca dataset, which is based on extended FLAN data from the Orca paper. The set has already been used to create strong model checkpoints with preference tuning.

What you can do with it

  • Language model alignment via DPO
  • Fine-tuning chat models to follow instructions and reason
  • Training reward models to evaluate response quality
Tasks
Text classification, Token classification, Table question answering, Question answering, Zero shot classification, Summarization
Languages
English
Size
10K–100K rows
Downloads this month
6.7K
Likes
32

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("HuggingFaceH4/orca_dpo_pairs")