Dataset · visheratin
RealWorldQA spatial understanding benchmark by xAI
visheratin/realworldqa
765 real-world photos with questions and answers by xAI for evaluating multimodal models.
What's inside
A benchmark released by xAI alongside the Grok-1.5 Vision announcement to assess basic real-world spatial understanding in VLMs. The English-language set includes 765 images, each accompanied by a question and an easily verifiable answer.
What you can do with it
- Evaluating spatial reasoning in multimodal models (VQA)
- Comparing visual models on challenging everyday scenes for AI
- Tasks
- Visual question answering
- Languages
- English
- Size
- up to 1K rows
- Downloads this month
- 237
- Likes
- 39
Models trained on it
How to load it
from datasets import load_dataset
ds = load_dataset("visheratin/realworldqa")