Dataset · We-Math
We-Math visual math problem benchmark (ACL 2025)
We-Math/We-Math
6.5k visual math problems with hierarchical knowledge annotation for evaluating VLM reasoning.
What's inside
The benchmark includes 6.5k English math problems with images and text presented at ACL 2025. The problems are carefully curated and categorized into 67 hierarchical concepts and 5 levels of knowledge granularity to test multimodal model reasoning logic rather than just the final answer.
What you can do with it
- Deep evaluation of mathematical capabilities and reasoning chains of multimodal LLMs
- Diagnosing gaps in specific geometry and visual mathematics sections
- Tasks
- Question answering, Text generation
- Languages
- English
- Size
- 1K–10K rows
- Downloads this month
- 4.9K
- Likes
- 35
How to load it
from datasets import load_dataset
ds = load_dataset("We-Math/We-Math")