Dataset · sled-umich
VLM multi-object hallucination benchmark
sled-umich/ROPE
Dataset for evaluating and analyzing hallucinations in vision-language models on complex scenes.
What's inside
The English-language dataset is built on annotated images from MSCOCO-Panoptic and ADE20K. It contains between 10,000 and 100,000 examples divided into several subsets linked to semantic object segmentation. It is designed to test how models invent non-existent objects in images.
What you can do with it
- Test multimodal models (VLMs) for hallucination tendencies
- Evaluate visual question answering (VQA) quality in multi-object scenes
- Tasks
- Question answering, Text classification
- Languages
- English
- Size
- 10K–100K rows
- Downloads this month
- 7.2K
- Likes
- 6
How to load it
from datasets import load_dataset
ds = load_dataset("sled-umich/ROPE")