Dataset · ONE-Lab
Video benchmark for multimodal AI agents in interfaces
ONE-Lab/GUI-World
Dataset featuring videos and questions to evaluate multimodal models in dynamic GUI environments.
What's inside
Contains between 10k and 100k English examples featuring screen recordings of graphical user interface interactions and annotations. Annotations cover six GUI usage scenarios and eight types of interface questions. Designed for evaluating modern ImageLLMs and VideoLLMs on multi-step tasks.
What you can do with it
- Evaluating multimodal models and AI agents in interface navigation
- Training VideoLLMs to understand dynamic on-screen actions
- Tasks
- Question answering, Text generation, Video text → text
- Languages
- English
- Size
- 10K–100K rows
- Downloads this month
- 20K
- Likes
- 46
How to load it
from datasets import load_dataset
ds = load_dataset("ONE-Lab/GUI-World")