Dataset · AnalystAIlab
AI assistant failure examples in everyday tasks
AnalystAIlab/ai-assistant-failure-cases
A small collection of everyday queries where popular AI assistants made notable errors.
What's inside
The repository contains fewer than 1,000 entries in text and tabular formats in English, collected by the analyst.ai project. Each case contains a real-world everyday prompt where a consumer AI assistant failed (plus one case where no one failed). The authors warn that the dataset is created to demonstrate types of failures rather than to calculate overall model rankings.
What you can do with it
- Manual testing and regression checks for custom AI assistants on complex cases
- Analysis of typical hallucinations and LLM errors in everyday scenarios
- Preparing edge-case prompts to evaluate model reliability
- Tasks
- Text generation, Question answering
- Languages
- English
- Size
- up to 1K rows
- Downloads this month
- 212
- Likes
- 11
How to load it
from datasets import load_dataset
ds = load_dataset("AnalystAIlab/ai-assistant-failure-cases")