Skip to content

Dataset · ServiceNow

GUI agent trajectories for OS LLM judge evaluation

ServiceNow/AgentHorizon

Benchmark from ServiceNow with screenshots and long computer agent action chains to evaluate judge models.

What's inside

The dataset contains between 100k and 1M records of GUI agent operations attempting complex multi-step desktop tasks (typically 100 to 300+ steps). Each example includes a text instruction, action sequence, screen screenshots, and a gold success or failure label for the task execution.

What you can do with it

  • Evaluating and calibrating LLM judges determining computer agent task success
  • Training reward models for graphical user interface automation systems
Size
100K–1M rows
Downloads this month
81K
Likes
4

How to load it

from datasets import load_dataset

ds = load_dataset("ServiceNow/AgentHorizon")