Skip to content

Dataset · xlangai

Gated task classes for OSWorld V2 benchmark

xlangai/osworld_v2_tasks

Official Python classes with task setup and evaluation logic for testing computer AI agents.

What's inside

The dataset includes under 1 thousand entries in text and table formats, representing the official task_*.py files for the second version of the OSWorld benchmark. Access to the repository is gated to prevent data leakage and ensure tested agents cannot find answers or verification code on the open web during task execution. Loading is performed via the huggingface_hub utility combined with the main GitHub repository.

What you can do with it

  • Evaluating autonomous GUI and OS agents in real-world OS workflows
  • Validating action-planning models without the risk of answer leakage into the network
  • Reproducing results on the current OSWorld V2 standard
Size
up to 1K rows
Downloads this month
51.3K
Likes
56
Access
on request on Hugging Face

How to load it

from datasets import load_dataset

ds = load_dataset("xlangai/osworld_v2_tasks")