Dataset · mlabonne
Preference mixture for ORPO and DPO
mlabonne/orpo-dpo-mix-40k
40k examples from several DPO datasets for preference alignment training.
What's inside
mlabonne/orpo-dpo-mix-40k is a mixture designed for ORPO or DPO training. It is compiled from several high-quality DPO sets with selected responses filtered by scores, such as Capybara-Preferences (7,424 examples), distilabel-intel-orca-dpo-pairs (2,299), and ultrafeedback-binarized-preferences-cleaned. The data is textual, in English, with a volume of 10k to 100k examples.
What you can do with it
- Train Llama 3 or LoRA via ORPO
- Perform DPO alignment of an assistant
- Compare preference selection strategies
- Tasks
- Text generation
- Languages
- English
- Size
- 10K–100K rows
- Downloads this month
- 729
- Likes
- 318
Models trained on it
- mlabonne/NeuralDaredevil-8B-abliteratedText generation 15.9K
- Kukedlc/NeuralLLaMa-3-8b-ORPO-v0.3Text generation 9.3K
- Danielbrdz/Barcenas-14b-Phi-3-medium-ORPOText generation 9.2K
- dfurman/Llama-3-8B-Orpo-v0.1Text generation 9.2K
- swap-uniba/LLaMAntino-3-ANITA-8B-Inst-DPO-ITAText generation 3.5K
- mradermacher/LLaMAntino-3-ANITA-8B-Inst-DPO-ITA-i1-GGUF 2.2K
- QuantFactory/NeuralDaredevil-8B-abliterated-GGUFText generation 2.2K
- anakin87/gemma-2-2b-neogenesis-itaText generation 2K
How to load it
from datasets import load_dataset
ds = load_dataset("mlabonne/orpo-dpo-mix-40k")