Dataset · prometheus-eval
Training data for LLM judges on long-form responses
prometheus-eval/Preference-Collection
Large English corpus for training open models to evaluate detailed texts.
What's inside
Includes 100K to 1M English examples designed to train language models in detailed evaluation of long responses. The project addresses the weak correlation between open models and human or GPT-4-tier proprietary system evaluations.
What you can do with it
- SFT training of open evaluator models (LLM-as-a-Judge) to check long generations
- Building reward models and feedback pipelines for LLM alignment
- Tasks
- Text generation, Text classification
- Languages
- English
- Size
- 100K–1M rows
- Downloads this month
- 434
- Likes
- 41
Models trained on it
- prometheus-eval/prometheus-7b-v2.0Text generation 2.6K
- prometheus-eval/prometheus-7b-v2.0-GGUFText generation 2.3K
- mradermacher/prometheus-2-llama-3-8b-i1-GGUF 1.2K
- mradermacher/prometheus-7b-v2.0-GGUF 929
- AlekseiPravdin/prometheus-7b-v2_0-ggufText generation 873
- prometheus-eval/prometheus-8x7b-v2.0Text generation 762
- mradermacher/prometheus-8x7b-v2.0-i1-GGUF 707
- mradermacher/prometheus-2-llama-3-8b-GGUF 478
How to load it
from datasets import load_dataset
ds = load_dataset("prometheus-eval/Preference-Collection")