Skip to content

Dataset · Qwen

World modeling benchmark for LLM agents

Qwen/AgentWorldBench

Evaluation of linguistic world models using agent trajectories and ground-truth observations from real environments.

What's inside

The benchmark is compiled from real observations of frontier model trajectories on Tool Decathlon, Terminal-Bench 1.0/2.0, and OSWorld-Verified. Each example is paired with a ground-truth observation from execution in the environment, enabling reference-grounded scoring. Modalities include tabular and text in English, with a size of 1K–10K.

What you can do with it

  • Evaluating LLM agents on action consequence prediction
  • Testing world modeling within an agentic system
  • Comparing models using reference-grounded scoring
Tasks
Text generation
Languages
English
Size
1K–10K rows
Downloads this month
1.3K
Likes
110

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("Qwen/AgentWorldBench")