Skip to content

Dataset · basicv8vc

SimpleQA benchmark for factual accuracy evaluation

basicv8vc/SimpleQA

OpenAI collection of short factual English questions for evaluating LLM truthfulness.

What's inside

The dataset is a text benchmark containing 1,000 to 10,000 short questions requiring precise factual answers in English. Materials are sourced from the openai/simple-evals repository and are designed to measure large language models' ability to provide concise and reliable answers.

What you can do with it

  • Measuring hallucination levels and LLM factual accuracy
  • Comparing models on short reference query responses
  • Evaluating RAG knowledge base retrieval system quality
Tasks
Question answering
Languages
English
Size
1K–10K rows
Downloads this month
4.8K
Likes
34

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("basicv8vc/SimpleQA")