Skip to content

Dataset · prometheus-eval

Training data for LLM judges on long-form responses

prometheus-eval/Preference-Collection

Large English corpus for training open models to evaluate detailed texts.

What's inside

Includes 100K to 1M English examples designed to train language models in detailed evaluation of long responses. The project addresses the weak correlation between open models and human or GPT-4-tier proprietary system evaluations.

What you can do with it

  • SFT training of open evaluator models (LLM-as-a-Judge) to check long generations
  • Building reward models and feedback pipelines for LLM alignment
Tasks
Text generation, Text classification
Languages
English
Size
100K–1M rows
Downloads this month
434
Likes
41

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("prometheus-eval/Preference-Collection")