Dataset · ibm-research
Knowledge conflict benchmark for LLMs
ibm-research/Wikipedia_contradict_benchmark
A dataset of 253 expert examples for evaluating LLMs under real-world knowledge conflict conditions in text.
What's inside
Includes 253 high-quality manually annotated instances in English. Created to test the performance of large language models when retrieving and integrating data from fragments with real contradictions.
What you can do with it
- Evaluating LLM robustness to conflicting data in RAG systems
- Testing hallucinations and knowledge conflict processing
- Tasks
- Question answering
- Languages
- English
- Size
- up to 1K rows
- Downloads this month
- 2.4K
- Likes
- 30
How to load it
from datasets import load_dataset
ds = load_dataset("ibm-research/Wikipedia_contradict_benchmark")