Dataset · VLLMs
Multi-image understanding benchmark for VLMs
VLLMs/MIRB
English visual question answering dataset for evaluating multimodal models' ability to analyze multiple images simultaneously.
What's inside
Contains between 1,000 and 10,000 examples with images in JPG format and corresponding questions in JSON files in English. Tasks are grouped into categories—such as analogy search or code analysis—and test perception, knowledge, and multi-step reasoning across multiple images simultaneously.
What you can do with it
- Testing multimodal models (VLMs) on tasks comparing and linking multiple images
- Evaluating multi-step visual reasoning skills (multi-hop reasoning)
- Comparing the quality of complex visual context understanding across different VLMs
- Tasks
- Question answering
- Languages
- English
- Size
- 1K–10K rows
- Downloads this month
- 1.2K
- Likes
- 15
How to load it
from datasets import load_dataset
ds = load_dataset("VLLMs/MIRB")