Dataset · prometheus-eval
Long answer evaluation feedback dataset
prometheus-eval/Feedback-Collection
English feedback texts to train open LLMs in fine-grained long answer evaluation.
What's inside
A fine-grained evaluation set: English feedback texts with 10k to 100k examples. Designed for fine-tuning open language models on long answer evaluation; the card notes that without such training, they correlate poorly with humans and GPT-4.
What you can do with it
- Training the Prometheus judge model
- LoRA for LLM-as-a-judge
- Checking evaluation correlation with humans and GPT-4
- Tasks
- Text generation, Text classification
- Languages
- English
- Size
- 10K–100K rows
- Downloads this month
- 971
- Likes
- 121
Models trained on it
- prometheus-eval/prometheus-7b-v2.0Text generation 2.6K
- prometheus-eval/prometheus-7b-v2.0-GGUFText generation 2.3K
- mradermacher/prometheus-2-llama-3-8b-i1-GGUF 1.2K
- mradermacher/prometheus-7b-v2.0-GGUF 929
- AlekseiPravdin/prometheus-7b-v2_0-ggufText generation 873
- prometheus-eval/prometheus-8x7b-v2.0Text generation 762
- mradermacher/prometheus-8x7b-v2.0-i1-GGUF 707
- mradermacher/prometheus-2-llama-3-8b-GGUF 478
How to load it
from datasets import load_dataset
ds = load_dataset("prometheus-eval/Feedback-Collection")