Dataset · google
Google short factual questions benchmark
google/simpleqa-verified
1,000 verified English queries from Google DeepMind for precise evaluation of LLM factual knowledge.
What's inside
The dataset includes 1,000 English prompts developed by researchers from Google DeepMind and Google Research. Created specifically for reliable evaluation of language models' parametric knowledge and their ability to provide short, factually accurate answers to direct questions.
What you can do with it
- Verifying factual accuracy and erudition of language models
- Benchmarking LLMs for tendency to hallucinate answers on direct questions
- Tasks
- Question answering
- Languages
- English
- Size
- 1K–10K rows
- Downloads this month
- 9.4K
- Likes
- 56
How to load it
from datasets import load_dataset
ds = load_dataset("google/simpleqa-verified")