Dataset · argilla
UltraFeedback binarized preferences
argilla/ultrafeedback-binarized-preferences-cleaned
Cleaned preference pairs from UltraFeedback for language model fine-tuning.
What's inside
English tabular and text dataset containing 10k to 100k examples. This is a new version of UltraFeedback binarization based on average preference scores recommended by Argilla for fine-tuning. The data has been cleaned.
What you can do with it
- RLHF/DPO fine-tuning
- Reward model training
- Evaluating LLM response quality
- Tasks
- Text generation
- Languages
- English
- Size
- 10K–100K rows
- Downloads this month
- 10.7K
- Likes
- 176
Models trained on it
- wenbopan/Faro-Yi-9B-DPOText generation 9.4K
- wenbopan/Faro-Yi-34B-DPOText generation 9.2K
- MoxoffSrL/Moxoff-Phi3Mini-DPOText generation 3.3K
- MoxoffSrL/Moxoff-Phi3Mini-PPOText generation 3.3K
- MoxoffSrL/Moxoff-Phi3Mini-ORPOText generation 3.3K
- MoxoffSrL/Moxoff-Phi3Mini-KTOText generation 3.3K
- TheBloke/dolphin-2.6-mistral-7B-dpo-GGUF 1.5K
- birgermoell/oellm-9b-256k-stage5-a4b-tool-rl-step48Text generation 1.4K
How to load it
from datasets import load_dataset
ds = load_dataset("argilla/ultrafeedback-binarized-preferences-cleaned")