Dataset · ArtificialAnalysis
LLM factuality and hallucination benchmark
ArtificialAnalysis/AA-Omniscience-Public
600 questions across various knowledge domains to evaluate factual accuracy and LLM hallucination tendency.
What's inside
Contains 600 text questions from a wide range of subject domains. Built as a benchmark to test large language models' (LLMs) ability to accurately reproduce facts and handle knowledge gaps correctly. Annotation covers question-answering and document processing tasks.
What you can do with it
- Evaluating LLM hallucination tendency on cross-domain facts
- Comparing parametric knowledge accuracy across different models
- Tasks
- Question answering
- Size
- up to 1K rows
- Downloads this month
- 18.2K
- Likes
- 57
How to load it
from datasets import load_dataset
ds = load_dataset("ArtificialAnalysis/AA-Omniscience-Public")