Skip to content

Dataset · allenai

Point annotation on images via text queries

allenai/pixmo-points

Millions of pairs of images, object descriptions, and point coordinates for training models to point out details.

What's inside

The dataset contains from 1 million to 10 million records consisting of images, referring expressions (descriptions of objects or regions), and precise point coordinates that these descriptions point to. The data was manually collected by human annotators and contains a wide variety of queries, including frequent object categories (more than 10 mentions). The dataset is part of the PixMo series and was used for training Molmo multimodal models in the skill of precise pointing to image elements.

What you can do with it

  • Training vision-language models to locate and point at specific objects via text queries
  • Creating UI agents and visual navigation systems that need to click on required screen elements
Size
1M–10M rows
Downloads this month
2.4K
Likes
49

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("allenai/pixmo-points")