Dataset · google
Visual reasoning benchmark in polar coordinates
google/polaris-bench
Official benchmark from Google DeepMind and Stanford for evaluating MLLM visual reasoning in polar coordinates.
What's inside
Dataset for evaluating multimodal language models (MLLM) in English, including 10k to 100k image-text pairs. Tasks are formatted as visual question-answering (VQA) and multiple-choice tests checking models' ability to reason in polar coordinates instead of standard Cartesian ones.
What you can do with it
- Evaluating spatial reasoning of multimodal LLMs without relying on Cartesian templates
- Comparing and benchmarking open and proprietary VLM models
- Testing robustness of visual encoders to non-standard coordinate systems
- Tasks
- Visual question answering, Multiple choice
- Languages
- English
- Size
- 10K–100K rows
- Downloads this month
- 627
- Likes
- 7
How to load it
from datasets import load_dataset
ds = load_dataset("google/polaris-bench")