Skip to content

Dataset · KRAFTON

VLM fine-grained visual comparison benchmark

KRAFTON/VLM-SubtleBench

A collection of images and English questions to test VLM ability to find subtle differences in similar pictures.

What's inside

The dataset from KRAFTON includes between 10k and 100k records with images and visual question answering (VQA) in English. Unlike standard tests with obvious differences, it features visually similar frames with barely noticeable distinctions. The task is designed to evaluate fine comparative analysis skills in vision-language models.

What you can do with it

  • Testing VLM ability to find minor discrepancies between two nearly identical images
  • Evaluating models for industrial anomaly detection, medical imaging, and aerial photography tasks
Tasks
Visual question answering, Image → text
Languages
English
Size
10K–100K rows
Downloads this month
29.9K
Likes
7

How to load it

from datasets import load_dataset

ds = load_dataset("KRAFTON/VLM-SubtleBench")