Skip to content

Dataset · cais

Curated Benchmark of 1,000 Complex Multimodal Questions

cais/hle-diamond

A filtered and community-verified subset of 1,000 questions from the Humanity's Last Exam benchmark.

What's inside

The dataset is a multimodal collection of text and visual questions from the Center for AI Safety and Scale AI. It includes 1,000 questions that underwent a year of curation and refinement with the research community. The authors request not to redistribute or re-upload these data publicly to preserve evaluation integrity.

What you can do with it

  • Testing multimodal LLMs on extremely complex tasks
  • Comparing reasoning models on a curated reference benchmark
Size
1K–10K rows
Downloads this month
8.5K
Likes
66
Access
on request on Hugging Face

How to load it

from datasets import load_dataset

ds = load_dataset("cais/hle-diamond")