Dataset · lmms-lab-encoder
POPE Benchmark for Multimodal Hallucinations
lmms-lab-encoder/POPE
Formatted version of the POPE test (10k-100k examples) for evaluating object hallucinations in VLMs.
What's inside
The dataset includes image-text question pairs ranging from 10 to 100 thousand records, sourced from a well-known research paper on hallucination evaluation. Data is specially repackaged for running in the lmms-eval evaluation pipeline.
What you can do with it
- Testing multimodal models for the tendency to hallucinate non-existent objects in photos
- Comparing visual perception quality of different VLMs within the lmms-eval framework
- Size
- 10K–100K rows
- Downloads this month
- 32.3K
- Likes
- 22
How to load it
from datasets import load_dataset
ds = load_dataset("lmms-lab-encoder/POPE")