Skip to content

Dataset · We-Math

We-Math visual math problem benchmark (ACL 2025)

We-Math/We-Math

6.5k visual math problems with hierarchical knowledge annotation for evaluating VLM reasoning.

What's inside

The benchmark includes 6.5k English math problems with images and text presented at ACL 2025. The problems are carefully curated and categorized into 67 hierarchical concepts and 5 levels of knowledge granularity to test multimodal model reasoning logic rather than just the final answer.

What you can do with it

  • Deep evaluation of mathematical capabilities and reasoning chains of multimodal LLMs
  • Diagnosing gaps in specific geometry and visual mathematics sections
Tasks
Question answering, Text generation
Languages
English
Size
1K–10K rows
Downloads this month
4.9K
Likes
35

How to load it

from datasets import load_dataset

ds = load_dataset("We-Math/We-Math")