Skip to content

Dataset · Lin-Chen

Data-Leakage-Free Multimodal Benchmark

Lin-Chen/MMStar

Curated English visual-text tasks for fair evaluation of large multimodal models.

What's inside

The dataset contains image-text examples in multiple-choice question format in English. The volume ranges from 1 to 10 thousand records. Samples were specifically selected so that models cannot answer without analyzing the image and to eliminate intersections with LLM training sets.

What you can do with it

  • Evaluating vision-language model (VLM) quality
  • Testing models for visual context dependency and absence of data leakage
Tasks
Multiple choice, Question answering, Visual question answering
Languages
English
Size
1K–10K rows
Downloads this month
26.2K
Likes
55

How to load it

from datasets import load_dataset

ds = load_dataset("Lin-Chen/MMStar")