Skip to content

Dataset · domenicrosati

Benchmark for checking LLM answer truthfulness

domenicrosati/TruthfulQA

817 adversarial English questions to evaluate neural networks' propensity for misconceptions.

What's inside

The benchmark contains 817 English questions across 38 thematic categories, including medicine, law, finance, and politics. The questions are designed to provoke popular human myths and false beliefs. The dataset allows checking whether a model relies on facts or reproduces widespread misconceptions.

What you can do with it

  • Evaluating language models' propensity for hallucinations and repeating popular myths
  • Comparing the effectiveness of alignment methods (SFT, RLHF) to improve factual accuracy
Tasks
Question answering
Languages
English
Size
up to 1K rows
Downloads this month
1.3K
Likes
55

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("domenicrosati/TruthfulQA")