Dataset · UCLNLP
Adversarial questions for testing text comprehension
UCLNLP/adversarial_qa
Three reading comprehension question sets created to identify weaknesses in language models.
What's inside
An English text dataset for Reading Comprehension tasks, collected with neural network participation in a labeling loop (BiDAF, BERTLarge, and RoBERTaLarge). It consists of three separate parts of 12,000 examples each (10,000 for training, 1,000 for validation, and 1,000 for testing). The questions are selected to confuse baseline models when searching for an answer in the text.
What you can do with it
- Testing machine reading models' robustness to complex and tricky questions
- Fine-tuning QA systems to improve the accuracy of extracting answers from text
- Comparing transformer architectures on adversarial examples
- Tasks
- Question answering
- Languages
- English
- Size
- 10K–100K rows
- Downloads this month
- 5.2K
- Likes
- 44
Models trained on it
How to load it
from datasets import load_dataset
ds = load_dataset("UCLNLP/adversarial_qa")