Dataset · Anthropic
Evaluations written by language models
Anthropic/model-written-evals
A collection of eval tasks generated by LLMs to test model behavior.
What's inside
The repository contains datasets written by language models for an Anthropic paper on discovering LLM behaviors. They are intended for studying the quality and properties of model-generated data and for evaluating other models. Tasks include multiple-choice, zero-shot classification, and question-answering in English, with a volume of 1k to 10k items.
What you can do with it
- Evaluate persona, sycophancy, and other LLM behaviors
- Test zero-shot classifiers
- Use as a model benchmark
- Tasks
- Multiple choice, Zero shot classification, Question answering
- Languages
- English
- Size
- 1K–10K rows
- Downloads this month
- 5K
- Likes
- 71
Models trained on it
- tasksource/deberta-small-long-nliZero shot classification 9.3K
- sileod/deberta-v3-base-tasksource-nliZero shot classification 8K
- sileod/deberta-v3-large-tasksource-nliZero shot classification 1.8K
- tasksource/deberta-base-long-nliZero shot classification 1.4K
- sileod/deberta-v3-xsmall-tasksource-nliZero shot classification 240
- or4cl3ai/Aiden_t5Text generation 181
- ITLL/deberta-v3-base-tasksource-nliZero shot classification 41
- zeroMN/SHMTAudio text → text 40
How to load it
from datasets import load_dataset
ds = load_dataset("Anthropic/model-written-evals")