Skip to content

Dataset · MMMU

MMMU-Pro multimodal benchmark

MMMU/MMMU_Pro

Multidisciplinary visual and textual questions for multimodal understanding evaluation.

What's inside

The dataset contains question-answering, visual-question-answering, and multiple-choice tasks with English images and text. This is the MMMU-Pro version—a more robust cross-disciplinary benchmark for multimodal models; the card includes links to the leaderboard, paper, and GitHub. Size ranges from 1K to 10K examples.

What you can do with it

  • Evaluating multimodal LLMs on cross-disciplinary questions
  • Testing vision-language models in multiple-choice format
  • Comparing models on the MMMU-Pro leaderboard
Tasks
Question answering, Visual question answering, Multiple choice
Languages
English
Size
1K–10K rows
Downloads this month
40.4K
Likes
103

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("MMMU/MMMU_Pro")