Dataset · ServiceNow
GUI agent trajectories for OS LLM judge evaluation
ServiceNow/AgentHorizon
Benchmark from ServiceNow with screenshots and long computer agent action chains to evaluate judge models.
What's inside
The dataset contains between 100k and 1M records of GUI agent operations attempting complex multi-step desktop tasks (typically 100 to 300+ steps). Each example includes a text instruction, action sequence, screen screenshots, and a gold success or failure label for the task execution.
What you can do with it
- Evaluating and calibrating LLM judges determining computer agent task success
- Training reward models for graphical user interface automation systems
- Size
- 100K–1M rows
- Downloads this month
- 81K
- Likes
- 4
How to load it
from datasets import load_dataset
ds = load_dataset("ServiceNow/AgentHorizon")