Dataset · Hothan
Multimodal benchmark of Olympiad scientific problems
Hothan/OlympiadBench
Complex Olympiad-level problems with images in English and Chinese for evaluating model scientific reasoning.
What's inside
The benchmark includes 1 to 10 thousand Olympiad-level problems presented as text and images (including physics problems). The data is bilingual in English and Chinese. The card accompanies an ACL 2024 paper and contains an updated version with corrected image rendering.
What you can do with it
- Evaluate the ability of VLMs and LLMs to solve complex scientific and physical problems
- Benchmark multimodal reasoning in English and Chinese
- Tasks
- Question answering, Visual question answering
- Languages
- Chinese, English
- Size
- 1K–10K rows
- Downloads this month
- 21.8K
- Likes
- 52
How to load it
from datasets import load_dataset
ds = load_dataset("Hothan/OlympiadBench")