Skip to content

Dataset · ONE-Lab

Video benchmark for multimodal AI agents in interfaces

ONE-Lab/GUI-World

Dataset featuring videos and questions to evaluate multimodal models in dynamic GUI environments.

What's inside

Contains between 10k and 100k English examples featuring screen recordings of graphical user interface interactions and annotations. Annotations cover six GUI usage scenarios and eight types of interface questions. Designed for evaluating modern ImageLLMs and VideoLLMs on multi-step tasks.

What you can do with it

  • Evaluating multimodal models and AI agents in interface navigation
  • Training VideoLLMs to understand dynamic on-screen actions
Tasks
Question answering, Text generation, Video text → text
Languages
English
Size
10K–100K rows
Downloads this month
20K
Likes
46

How to load it

from datasets import load_dataset

ds = load_dataset("ONE-Lab/GUI-World")