Dataset · cais
Humanity's Last Exam benchmark
cais/hle
Multimodal benchmark of 2500 difficult questions across dozens of academic topics.
What's inside
This is HLE, a multimodal benchmark with 2500 questions across dozens of subjects, conceived as a difficult closed academic test at the frontier of human knowledge. Includes images and text; created by Center for AI Safety and Scale AI. Authors ask not to publish, re-upload, or distribute the dataset.
What you can do with it
- Testing frontier multimodal LLMs
- Evaluating complex academic reasoning
- Comparing models on a closed benchmark
- Size
- 1K–10K rows
- Downloads this month
- 81.2K
- Likes
- 1.2K
- Access
- on request on Hugging Face
Models trained on it
How to load it
from datasets import load_dataset
ds = load_dataset("cais/hle")