openai/gdpval
OpenAI's dataset for benchmarking AI on real-world work tasks
2 min read
An official dataset from OpenAI designed to measure how well AI models handle economically valuable tasks from real-world practice. It packs 220 cases across 44 professions, spanning everything from law and politics to film and literature. Unlike abstract academic benchmarks, every task pairs a text prompt with supporting reference files (documents, images, and videos) that models must work through together. It is a solid tool for LLM and multimodal assistant developers looking to run honest evaluations of practical, on-the-job skills.
Key features
- 220 practical tasks: Covers 44 real professions to simulate genuine day-to-day workloads.
- File-based context: Every case pairs a text prompt with supporting reference materials.
- No artificial censorship: Retains tasks involving sensitive, political, or 18+ topics since lawyers, screenwriters, and other professionals regularly deal with them in practice.
- Training-leak protection: Includes a canary string (a unique marker used to detect if benchmark data leaks into training sets).
What you can do
- Test your LLM or VLM: See how well a model executes multi-part work assignments grounded in attached files.
- Evaluate AI agents: Use the tasks as a testbed for systems that need to analyze documents and media before making decisions.
- Compare different models: Run your own benchmarks pitting proprietary models against open-source ones on applied business scenarios.
Example
from datasets import load_dataset
dataset = load_dataset("openai/gdpval")
print(dataset)
Read more
DetailsSize: 220 tasks across 44 professions (size n<1K).
Contents: Text prompts and reference files (including images and videos featuring AI-generated or consented individuals).
Notes: Includes brand mentions for research purposes, alongside political and NSFW content reflecting the realities of the covered professions.








Comments