Skip to content

Dataset · mlabonne

Preference mixture for ORPO and DPO

mlabonne/orpo-dpo-mix-40k

40k examples from several DPO datasets for preference alignment training.

What's inside

mlabonne/orpo-dpo-mix-40k is a mixture designed for ORPO or DPO training. It is compiled from several high-quality DPO sets with selected responses filtered by scores, such as Capybara-Preferences (7,424 examples), distilabel-intel-orca-dpo-pairs (2,299), and ultrafeedback-binarized-preferences-cleaned. The data is textual, in English, with a volume of 10k to 100k examples.

What you can do with it

  • Train Llama 3 or LoRA via ORPO
  • Perform DPO alignment of an assistant
  • Compare preference selection strategies
Tasks
Text generation
Languages
English
Size
10K–100K rows
Downloads this month
729
Likes
318

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("mlabonne/orpo-dpo-mix-40k")