Dataset · xlangai
Gated task classes for OSWorld V2 benchmark
xlangai/osworld_v2_tasks
Official Python classes with task setup and evaluation logic for testing computer AI agents.
What's inside
The dataset includes under 1 thousand entries in text and table formats, representing the official task_*.py files for the second version of the OSWorld benchmark. Access to the repository is gated to prevent data leakage and ensure tested agents cannot find answers or verification code on the open web during task execution. Loading is performed via the huggingface_hub utility combined with the main GitHub repository.
What you can do with it
- Evaluating autonomous GUI and OS agents in real-world OS workflows
- Validating action-planning models without the risk of answer leakage into the network
- Reproducing results on the current OSWorld V2 standard
- Size
- up to 1K rows
- Downloads this month
- 51.3K
- Likes
- 56
- Access
- on request on Hugging Face
How to load it
from datasets import load_dataset
ds = load_dataset("xlangai/osworld_v2_tasks")