Skip to content

Dataset · ArtificialAnalysis

LLM factuality and hallucination benchmark

ArtificialAnalysis/AA-Omniscience-Public

600 questions across various knowledge domains to evaluate factual accuracy and LLM hallucination tendency.

What's inside

Contains 600 text questions from a wide range of subject domains. Built as a benchmark to test large language models' (LLMs) ability to accurately reproduce facts and handle knowledge gaps correctly. Annotation covers question-answering and document processing tasks.

What you can do with it

  • Evaluating LLM hallucination tendency on cross-domain facts
  • Comparing parametric knowledge accuracy across different models
Tasks
Question answering
Size
up to 1K rows
Downloads this month
18.2K
Likes
57

How to load it

from datasets import load_dataset

ds = load_dataset("ArtificialAnalysis/AA-Omniscience-Public")