Dataset · basicv8vc
SimpleQA benchmark for factual accuracy evaluation
basicv8vc/SimpleQA
OpenAI collection of short factual English questions for evaluating LLM truthfulness.
What's inside
The dataset is a text benchmark containing 1,000 to 10,000 short questions requiring precise factual answers in English. Materials are sourced from the openai/simple-evals repository and are designed to measure large language models' ability to provide concise and reliable answers.
What you can do with it
- Measuring hallucination levels and LLM factual accuracy
- Comparing models on short reference query responses
- Evaluating RAG knowledge base retrieval system quality
- Tasks
- Question answering
- Languages
- English
- Size
- 1K–10K rows
- Downloads this month
- 4.8K
- Likes
- 34
Models trained on it
How to load it
from datasets import load_dataset
ds = load_dataset("basicv8vc/SimpleQA")