Dataset · argilla
Preference dataset based on Capybara dialogues
argilla/Capybara-Preferences
English multi-turn dialogues for LLM alignment and preference training (RLHF/DPO).
What's inside
Inside are dialogues in English collected based on the LDJnr/Capybara instructional dataset using the distilabel tool. The last assistant response was removed from the original conversations to turn the chat history up to the last user question into a base for preference evaluation. The set volume is between 10 and 100 thousand records in text and tabular form.
What you can do with it
- Fine-tuning language models using RLHF or DPO on multi-turn dialogues
- Training reward models to evaluate the quality of assistant responses
- Tasks
- Text generation
- Languages
- English
- Size
- 10K–100K rows
- Downloads this month
- 285
- Likes
- 48
Models trained on it
- mradermacher/mistral-7b-orpo-v5.0-GGUF 600
- nlee-208/inverted-orpo-mistral-capybaraText generation 93
- orpo-explorers/kaist-mistral-orpo-OHP-15k-Stratified-1-beta-0.2-2epoch-capybara-1epochText generation 9
- orpo-explorers/kaist-mistral-orpo-OHP-15k-Stratified-1-beta-0.2-1epoch-capybara-2epochText generation 8
How to load it
from datasets import load_dataset
ds = load_dataset("argilla/Capybara-Preferences")