Skip to content

Dataset · visheratin

RealWorldQA spatial understanding benchmark by xAI

visheratin/realworldqa

765 real-world photos with questions and answers by xAI for evaluating multimodal models.

What's inside

A benchmark released by xAI alongside the Grok-1.5 Vision announcement to assess basic real-world spatial understanding in VLMs. The English-language set includes 765 images, each accompanied by a question and an easily verifiable answer.

What you can do with it

  • Evaluating spatial reasoning in multimodal models (VQA)
  • Comparing visual models on challenging everyday scenes for AI
Tasks
Visual question answering
Languages
English
Size
up to 1K rows
Downloads this month
237
Likes
39

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("visheratin/realworldqa")