Skip to content

Dataset · Anthropic

Evaluations written by language models

Anthropic/model-written-evals

A collection of eval tasks generated by LLMs to test model behavior.

What's inside

The repository contains datasets written by language models for an Anthropic paper on discovering LLM behaviors. They are intended for studying the quality and properties of model-generated data and for evaluating other models. Tasks include multiple-choice, zero-shot classification, and question-answering in English, with a volume of 1k to 10k items.

What you can do with it

  • Evaluate persona, sycophancy, and other LLM behaviors
  • Test zero-shot classifiers
  • Use as a model benchmark
Tasks
Multiple choice, Zero shot classification, Question answering
Languages
English
Size
1K–10K rows
Downloads this month
5K
Likes
71

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("Anthropic/model-written-evals")