Dataset · lmms-lab-encoder
Combined MMMU multimodal benchmark
lmms-lab-encoder/MMMU
A unified version of the MMMU benchmark (10K–100K examples) with all subsets combined for evaluating multimodal models.
What's inside
This is a reformatted version of the well-known MMMU benchmark where all thematic subsets are merged into a single general dataset. It contains image-text pairs and is prepared for running in the lmms-eval pipeline.
What you can do with it
- Rapid comprehensive evaluation of large multimodal models (LMMs) via the lmms-eval framework
- Comparing vision-text models on academic tasks without manual subset stitching
- Size
- 10K–100K rows
- Downloads this month
- 22.6K
- Likes
- 7
How to load it
from datasets import load_dataset
ds = load_dataset("lmms-lab-encoder/MMMU")