Skip to content

Dataset · xai-org

RealWorldQA: Questions on real-world images

xai-org/RealworldQA

Benchmark of 700+ real-world images with questions and verifiable answers for multimodal models.

What's inside

RealWorldQA is a benchmark for real-world understanding. The initial version features over 700 anonymized images captured from vehicles and other real photos, each with a question and easily verifiable answer. The dataset was released by xAI and is mentioned in the Grok-1.5 Vision Preview announcement. Language is unspecified; format is image and text pairs.

What you can do with it

  • Evaluating VLMs on real-world scenes
  • Testing visual question answering
  • Comparing multimodal models on the benchmark
Size
up to 1K rows
Downloads this month
6.4K
Likes
130

How to load it

from datasets import load_dataset

ds = load_dataset("xai-org/RealworldQA")