Dataset · mhjiang0408
Scientific cover benchmark for multimodal LLMs
mhjiang0408/MAC_Bench
A dynamic English-language benchmark for assessing how well multimodal LLMs understand scientific journals and illustrations.
What's inside
It gathers English-language images and text, ranging from 1,000 to 10,000 examples dedicated to scientific journal covers. The data is structured as an updatable benchmark for verifying visual-textual comprehension in an academic setting.
What you can do with it
- Evaluating the quality of multimodal LLMs in image-text-to-text tasks
- Testing models on their comprehension of complex scientific illustrations and graphics
- Tasks
- Image text → text
- Languages
- English
- Size
- 1K–10K rows
- Downloads this month
- 50.5K
- Likes
- 1
How to load it
from datasets import load_dataset
ds = load_dataset("mhjiang0408/MAC_Bench")