Skip to content

Dataset · google

Google short factual questions benchmark

google/simpleqa-verified

1,000 verified English queries from Google DeepMind for precise evaluation of LLM factual knowledge.

What's inside

The dataset includes 1,000 English prompts developed by researchers from Google DeepMind and Google Research. Created specifically for reliable evaluation of language models' parametric knowledge and their ability to provide short, factually accurate answers to direct questions.

What you can do with it

  • Verifying factual accuracy and erudition of language models
  • Benchmarking LLMs for tendency to hallucinate answers on direct questions
Tasks
Question answering
Languages
English
Size
1K–10K rows
Downloads this month
9.4K
Likes
56

How to load it

from datasets import load_dataset

ds = load_dataset("google/simpleqa-verified")