Skip to content

Dataset · prometheus-eval

Long answer evaluation feedback dataset

prometheus-eval/Feedback-Collection

English feedback texts to train open LLMs in fine-grained long answer evaluation.

What's inside

A fine-grained evaluation set: English feedback texts with 10k to 100k examples. Designed for fine-tuning open language models on long answer evaluation; the card notes that without such training, they correlate poorly with humans and GPT-4.

What you can do with it

  • Training the Prometheus judge model
  • LoRA for LLM-as-a-judge
  • Checking evaluation correlation with humans and GPT-4
Tasks
Text generation, Text classification
Languages
English
Size
10K–100K rows
Downloads this month
971
Likes
121

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("prometheus-eval/Feedback-Collection")