Dataset · Lin-Chen
Data-Leakage-Free Multimodal Benchmark
Lin-Chen/MMStar
Curated English visual-text tasks for fair evaluation of large multimodal models.
What's inside
The dataset contains image-text examples in multiple-choice question format in English. The volume ranges from 1 to 10 thousand records. Samples were specifically selected so that models cannot answer without analyzing the image and to eliminate intersections with LLM training sets.
What you can do with it
- Evaluating vision-language model (VLM) quality
- Testing models for visual context dependency and absence of data leakage
- Tasks
- Multiple choice, Question answering, Visual question answering
- Languages
- English
- Size
- 1K–10K rows
- Downloads this month
- 26.2K
- Likes
- 55
How to load it
from datasets import load_dataset
ds = load_dataset("Lin-Chen/MMStar")