Dataset · lmsys
MT-bench Human Judgment Dataset
lmsys/mt_bench_human_judgments
3.3k expert pairwise preference annotations for six model responses to 80 MT-bench questions.
What's inside
The dataset contains 3.3k expert pairwise human preferences between responses from six models to 80 MT-bench questions. Models include GPT-4, GPT-3.5, Claude-v1, Vicuna-13B, Alpaca-13B, and LLaMA-13B. Annotators are primarily graduate students knowledgeable in the question domains; language is English.
What you can do with it
- Train a reward model on pairwise preferences
- Evaluate an LLM-as-a-judge against human evaluations
- Benchmark models on MT-bench questions
- Tasks
- Question answering
- Languages
- English
- Size
- 1K–10K rows
- Downloads this month
- 15.8K
- Likes
- 150
Models trained on it
How to load it
from datasets import load_dataset
ds = load_dataset("lmsys/mt_bench_human_judgments")