Skip to content

Dataset · Hothan

Multimodal benchmark of Olympiad scientific problems

Hothan/OlympiadBench

Complex Olympiad-level problems with images in English and Chinese for evaluating model scientific reasoning.

What's inside

The benchmark includes 1 to 10 thousand Olympiad-level problems presented as text and images (including physics problems). The data is bilingual in English and Chinese. The card accompanies an ACL 2024 paper and contains an updated version with corrected image rendering.

What you can do with it

  • Evaluate the ability of VLMs and LLMs to solve complex scientific and physical problems
  • Benchmark multimodal reasoning in English and Chinese
Tasks
Question answering, Visual question answering
Languages
Chinese, English
Size
1K–10K rows
Downloads this month
21.8K
Likes
52

How to load it

from datasets import load_dataset

ds = load_dataset("Hothan/OlympiadBench")