Skip to content

Dataset · google

Visual reasoning benchmark in polar coordinates

google/polaris-bench

Official benchmark from Google DeepMind and Stanford for evaluating MLLM visual reasoning in polar coordinates.

What's inside

Dataset for evaluating multimodal language models (MLLM) in English, including 10k to 100k image-text pairs. Tasks are formatted as visual question-answering (VQA) and multiple-choice tests checking models' ability to reason in polar coordinates instead of standard Cartesian ones.

What you can do with it

  • Evaluating spatial reasoning of multimodal LLMs without relying on Cartesian templates
  • Comparing and benchmarking open and proprietary VLM models
  • Testing robustness of visual encoders to non-standard coordinate systems
Tasks
Visual question answering, Multiple choice
Languages
English
Size
10K–100K rows
Downloads this month
627
Likes
7

How to load it

from datasets import load_dataset

ds = load_dataset("google/polaris-bench")