Skip to content

Dataset · lmms-lab-encoder

POPE Benchmark for Multimodal Hallucinations

lmms-lab-encoder/POPE

Formatted version of the POPE test (10k-100k examples) for evaluating object hallucinations in VLMs.

What's inside

The dataset includes image-text question pairs ranging from 10 to 100 thousand records, sourced from a well-known research paper on hallucination evaluation. Data is specially repackaged for running in the lmms-eval evaluation pipeline.

What you can do with it

  • Testing multimodal models for the tendency to hallucinate non-existent objects in photos
  • Comparing visual perception quality of different VLMs within the lmms-eval framework
Size
10K–100K rows
Downloads this month
32.3K
Likes
22

How to load it

from datasets import load_dataset

ds = load_dataset("lmms-lab-encoder/POPE")