Dataset · domenicrosati
Benchmark for checking LLM answer truthfulness
domenicrosati/TruthfulQA
817 adversarial English questions to evaluate neural networks' propensity for misconceptions.
What's inside
The benchmark contains 817 English questions across 38 thematic categories, including medicine, law, finance, and politics. The questions are designed to provoke popular human myths and false beliefs. The dataset allows checking whether a model relies on facts or reproduces widespread misconceptions.
What you can do with it
- Evaluating language models' propensity for hallucinations and repeating popular myths
- Comparing the effectiveness of alignment methods (SFT, RLHF) to improve factual accuracy
- Tasks
- Question answering
- Languages
- English
- Size
- up to 1K rows
- Downloads this month
- 1.3K
- Likes
- 55
Models trained on it
How to load it
from datasets import load_dataset
ds = load_dataset("domenicrosati/TruthfulQA")