Skip to content

Dataset · mhjiang0408

Scientific cover benchmark for multimodal LLMs

mhjiang0408/MAC_Bench

A dynamic English-language benchmark for assessing how well multimodal LLMs understand scientific journals and illustrations.

What's inside

It gathers English-language images and text, ranging from 1,000 to 10,000 examples dedicated to scientific journal covers. The data is structured as an updatable benchmark for verifying visual-textual comprehension in an academic setting.

What you can do with it

  • Evaluating the quality of multimodal LLMs in image-text-to-text tasks
  • Testing models on their comprehension of complex scientific illustrations and graphics
Tasks
Image text → text
Languages
English
Size
1K–10K rows
Downloads this month
50.5K
Likes
1

How to load it

from datasets import load_dataset

ds = load_dataset("mhjiang0408/MAC_Bench")