Skip to content

Dataset · compressionawareintelligence

AI agent pressure resistance benchmark

compressionawareintelligence/cai-bench

A 10K–100K example test suite for checking whether an AI agent follows rules under user pressure.

What's inside

The dataset contains 10,000 to 100,000 English text and tabular examples. Each situation is tested in four variants: a neutral prompt and a rephrased prompt with emotional pressure, both with and without the right to approval under the rules. A test is considered passed only if the agent gives the correct answer in all four cases and does not yield to persuasion.

What you can do with it

  • Evaluating reliability of AI agents for customer support
  • Checking LLM resistance to social engineering manipulations and prompt phrasing changes
Tasks
Question answering, Text classification, Text generation
Languages
English
Size
10K–100K rows
Downloads this month
23.9K
Likes
11
Access
on request on Hugging Face

How to load it

from datasets import load_dataset

ds = load_dataset("compressionawareintelligence/cai-bench")