Dataset · xai-org
RealWorldQA: Questions on real-world images
xai-org/RealworldQA
Benchmark of 700+ real-world images with questions and verifiable answers for multimodal models.
What's inside
RealWorldQA is a benchmark for real-world understanding. The initial version features over 700 anonymized images captured from vehicles and other real photos, each with a question and easily verifiable answer. The dataset was released by xAI and is mentioned in the Grok-1.5 Vision Preview announcement. Language is unspecified; format is image and text pairs.
What you can do with it
- Evaluating VLMs on real-world scenes
- Testing visual question answering
- Comparing multimodal models on the benchmark
- Size
- up to 1K rows
- Downloads this month
- 6.4K
- Likes
- 130
How to load it
from datasets import load_dataset
ds = load_dataset("xai-org/RealworldQA")