Dataset · ucinlp
DROP: Discrete reading benchmark
ucinlp/drop
96k paragraph-based questions to test arithmetic, counting, and sorting.
What's inside
The English-language DROP benchmark consists of 96k crowdsourced and adversarially created questions. Systems must resolve references in the question, potentially to multiple places, and perform discrete operations such as addition, counting, and sorting. The data is designed for reading comprehension and question answering.
What you can do with it
- Evaluate QA models on discrete operations
- Fine-tune reading comprehension models
- Compare LLMs on the DROP benchmark
- Tasks
- Question answering
- Languages
- English
- Size
- 10K–100K rows
- Downloads this month
- 14.6K
- Likes
- 72
Models trained on it
- saytes/SoT_DistilBERTText classification 45
- nielsr/nt5-small-rc1 26
- AdapterHub/roberta-base-pf-dropQuestion answering 20
- Avinash250325/T5BaseQuestionGenerationText generation 19
- AdapterHub/bert-base-uncased-pf-dropQuestion answering 14
- UKP-SQuARE/bert-base-uncased-pf-drop-onnxQuestion answering 0
- UKP-SQuARE/roberta-base-pf-drop-onnxQuestion answering 0
- neopolita/doc-to-lora-ministral-3b-2512Text generation 0
How to load it
from datasets import load_dataset
ds = load_dataset("ucinlp/drop")