Dataset · evayinhao
VLM spatial reasoning benchmark in mirror scenes
evayinhao/MIRA-Bench
Over 14,000 question-answer pairs on real photos with reflections for evaluating multimodal models.
What's inside
The MIRA-Bench benchmark includes 14,140 English question-answer pairs compiled for 1,011 images from 525 real mirror scenes. The release includes original annotations and 430 self-captured RGB photos. The dataset was created to test how well vision-language models (VLM) distinguish real objects from their reflections, understand space, and make decisions.
What you can do with it
- Testing vision-language models (VLM) for robustness against optical illusions and reflections
- Evaluating spatial reasoning skills in multimodal AI assistants
- Testing computer vision systems for robotics in complex interiors
- Tasks
- Visual question answering
- Languages
- English
- Size
- 10K–100K rows
- Downloads this month
- 803
- Likes
- 4
- Access
- on request on Hugging Face
How to load it
from datasets import load_dataset
ds = load_dataset("evayinhao/MIRA-Bench")