Skip to content

Dataset · evayinhao

VLM spatial reasoning benchmark in mirror scenes

evayinhao/MIRA-Bench

Over 14,000 question-answer pairs on real photos with reflections for evaluating multimodal models.

What's inside

The MIRA-Bench benchmark includes 14,140 English question-answer pairs compiled for 1,011 images from 525 real mirror scenes. The release includes original annotations and 430 self-captured RGB photos. The dataset was created to test how well vision-language models (VLM) distinguish real objects from their reflections, understand space, and make decisions.

What you can do with it

  • Testing vision-language models (VLM) for robustness against optical illusions and reflections
  • Evaluating spatial reasoning skills in multimodal AI assistants
  • Testing computer vision systems for robotics in complex interiors
Tasks
Visual question answering
Languages
English
Size
10K–100K rows
Downloads this month
803
Likes
4
Access
on request on Hugging Face

How to load it

from datasets import load_dataset

ds = load_dataset("evayinhao/MIRA-Bench")