Skip to content

Dataset · ibm-research

Knowledge conflict benchmark for LLMs

ibm-research/Wikipedia_contradict_benchmark

A dataset of 253 expert examples for evaluating LLMs under real-world knowledge conflict conditions in text.

What's inside

Includes 253 high-quality manually annotated instances in English. Created to test the performance of large language models when retrieving and integrating data from fragments with real contradictions.

What you can do with it

  • Evaluating LLM robustness to conflicting data in RAG systems
  • Testing hallucinations and knowledge conflict processing
Tasks
Question answering
Languages
English
Size
up to 1K rows
Downloads this month
2.4K
Likes
30

How to load it

from datasets import load_dataset

ds = load_dataset("ibm-research/Wikipedia_contradict_benchmark")