Skip to content

Dataset · VLLMs

Multi-image understanding benchmark for VLMs

VLLMs/MIRB

English visual question answering dataset for evaluating multimodal models' ability to analyze multiple images simultaneously.

What's inside

Contains between 1,000 and 10,000 examples with images in JPG format and corresponding questions in JSON files in English. Tasks are grouped into categories—such as analogy search or code analysis—and test perception, knowledge, and multi-step reasoning across multiple images simultaneously.

What you can do with it

  • Testing multimodal models (VLMs) on tasks comparing and linking multiple images
  • Evaluating multi-step visual reasoning skills (multi-hop reasoning)
  • Comparing the quality of complex visual context understanding across different VLMs
Tasks
Question answering
Languages
English
Size
1K–10K rows
Downloads this month
1.2K
Likes
15

How to load it

from datasets import load_dataset

ds = load_dataset("VLLMs/MIRB")