Dataset · MMMU
MMMU-Pro multimodal benchmark
MMMU/MMMU_Pro
Multidisciplinary visual and textual questions for multimodal understanding evaluation.
What's inside
The dataset contains question-answering, visual-question-answering, and multiple-choice tasks with English images and text. This is the MMMU-Pro version—a more robust cross-disciplinary benchmark for multimodal models; the card includes links to the leaderboard, paper, and GitHub. Size ranges from 1K to 10K examples.
What you can do with it
- Evaluating multimodal LLMs on cross-disciplinary questions
- Testing vision-language models in multiple-choice format
- Comparing models on the MMMU-Pro leaderboard
- Tasks
- Question answering, Visual question answering, Multiple choice
- Languages
- English
- Size
- 1K–10K rows
- Downloads this month
- 40.4K
- Likes
- 103
Models trained on it
How to load it
from datasets import load_dataset
ds = load_dataset("MMMU/MMMU_Pro")