Dataset · bevaya
Text-based UI element grounding benchmark
bevaya/ScreenSpot
Over 1,200 instructions and screenshots from iOS, Android, macOS, Windows, and the web for evaluating multimodal models.
What's inside
An evaluation benchmark for GUI grounding created by researchers from Nanjing University and Shanghai AI Laboratory. It contains over 1,200 English text instructions paired with annotated UI screenshots from mobile, desktop, and web systems. Designed to test how accurately multimodal models locate requested screen elements based on descriptions.
What you can do with it
- Evaluating multimodal models (LMMs) in interface navigation tasks
- Testing AI agents controlling computers or smartphones
- Tasks
- Text generation, Image → text
- Languages
- English
- Size
- 1K–10K rows
- Downloads this month
- 3.7K
- Likes
- 53
How to load it
from datasets import load_dataset
ds = load_dataset("bevaya/ScreenSpot")