Skip to content

Dataset · sled-umich

VLM multi-object hallucination benchmark

sled-umich/ROPE

Dataset for evaluating and analyzing hallucinations in vision-language models on complex scenes.

What's inside

The English-language dataset is built on annotated images from MSCOCO-Panoptic and ADE20K. It contains between 10,000 and 100,000 examples divided into several subsets linked to semantic object segmentation. It is designed to test how models invent non-existent objects in images.

What you can do with it

  • Test multimodal models (VLMs) for hallucination tendencies
  • Evaluate visual question answering (VQA) quality in multi-object scenes
Tasks
Question answering, Text classification
Languages
English
Size
10K–100K rows
Downloads this month
7.2K
Likes
6

How to load it

from datasets import load_dataset

ds = load_dataset("sled-umich/ROPE")