Skip to content

Dataset · cais

Humanity's Last Exam benchmark

cais/hle

Multimodal benchmark of 2500 difficult questions across dozens of academic topics.

What's inside

This is HLE, a multimodal benchmark with 2500 questions across dozens of subjects, conceived as a difficult closed academic test at the frontier of human knowledge. Includes images and text; created by Center for AI Safety and Scale AI. Authors ask not to publish, re-upload, or distribute the dataset.

What you can do with it

  • Testing frontier multimodal LLMs
  • Evaluating complex academic reasoning
  • Comparing models on a closed benchmark
Size
1K–10K rows
Downloads this month
81.2K
Likes
1.2K
Access
on request on Hugging Face

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("cais/hle")