Dataset · allenai
Long and detailed image descriptions for VLMs
allenai/pixmo-cap
Hundreds of thousands of images with detailed texts averaging 200 words and original speech transcripts.
What's inside
The dataset contains between 100 thousand and 1 million pairs of images and highly detailed text descriptions (averaging about 200 words). To create it, annotators voiced everything they saw in the image for 60-90 seconds, and then a Claude model turned the transcription into coherent text. The audio transcripts themselves are also included in the dataset. The set is part of the PixMo collection and was used in training the Molmo family of multimodal models.
What you can do with it
- Pre-training and fine-tuning (SFT) multimodal models (VLMs) for generating detailed descriptions
- Reducing hallucinations in vision-language models through training on detailed annotations
- Tasks
- Image → text
- Size
- 100K–1M rows
- Downloads this month
- 1.5K
- Likes
- 49
Models trained on it
How to load it
from datasets import load_dataset
ds = load_dataset("allenai/pixmo-cap")