Skip to content

Dataset · AI45Research

AI agent trajectory safety benchmark

AI45Research/ATBench

A set of long AI agent tool-use scenarios for evaluating and diagnosing agent safety.

What's inside

The repository collects full task execution trajectories by AI agents invoking external tools: a recent release includes 1,000 trajectories, while the first version of ATBench500 with 500 examples is also preserved. Data contains text, tables, and images, covers 2,084 available tools, and takes about 9 steps on average. Samples are balanced between safe and unsafe scenarios, and have undergone rule-based, LLM, and full manual filtering. Unsafe cases are labeled according to a taxonomy of three axes: source of risk, failure mode, and actual harm.

What you can do with it

  • Evaluating the safety and reliability of long agentic chains with tool invocations
  • Testing and diagnosing guardrails for AI agents
Size
1K–10K rows
Downloads this month
2.3K
Likes
46

How to load it

from datasets import load_dataset

ds = load_dataset("AI45Research/ATBench")