Dataset · AI45Research
AI agent trajectory safety benchmark
AI45Research/ATBench
A set of long AI agent tool-use scenarios for evaluating and diagnosing agent safety.
What's inside
The repository collects full task execution trajectories by AI agents invoking external tools: a recent release includes 1,000 trajectories, while the first version of ATBench500 with 500 examples is also preserved. Data contains text, tables, and images, covers 2,084 available tools, and takes about 9 steps on average. Samples are balanced between safe and unsafe scenarios, and have undergone rule-based, LLM, and full manual filtering. Unsafe cases are labeled according to a taxonomy of three axes: source of risk, failure mode, and actual harm.
What you can do with it
- Evaluating the safety and reliability of long agentic chains with tool invocations
- Testing and diagnosing guardrails for AI agents
- Size
- 1K–10K rows
- Downloads this month
- 2.3K
- Likes
- 46
How to load it
from datasets import load_dataset
ds = load_dataset("AI45Research/ATBench")