Skip to content

Dataset · truthfulqa

TruthfulQA: LLM truthfulness benchmark

truthfulqa/truthful_qa

817 questions across 38 categories to test whether a model generates truthful answers.

What's inside

TruthfulQA is a benchmark that measures how truthfully a language model answers questions. It includes 817 questions spanning 38 categories, including health, law, finance, and politics. The questions are crafted so that humans might answer falsely due to misconceptions, testing whether models avoid repeating learned falsehoods.

What you can do with it

  • Evaluating language model truthfulness
  • Testing QA models
  • Comparing models on the benchmark
Tasks
Multiple choice, Text generation, Question answering
Languages
English
Size
1K–10K rows
Downloads this month
96.7K
Likes
302

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("truthfulqa/truthful_qa")