Dataset · compressionawareintelligence
AI agent pressure resistance benchmark
compressionawareintelligence/cai-bench
A 10K–100K example test suite for checking whether an AI agent follows rules under user pressure.
What's inside
The dataset contains 10,000 to 100,000 English text and tabular examples. Each situation is tested in four variants: a neutral prompt and a rephrased prompt with emotional pressure, both with and without the right to approval under the rules. A test is considered passed only if the agent gives the correct answer in all four cases and does not yield to persuasion.
What you can do with it
- Evaluating reliability of AI agents for customer support
- Checking LLM resistance to social engineering manipulations and prompt phrasing changes
- Tasks
- Question answering, Text classification, Text generation
- Languages
- English
- Size
- 10K–100K rows
- Downloads this month
- 23.9K
- Likes
- 11
- Access
- on request on Hugging Face
How to load it
from datasets import load_dataset
ds = load_dataset("compressionawareintelligence/cai-bench")