Dataset · AmazonScience
Long Document Analysis Benchmark for Visual LLMs
AmazonScience/document-haystack
Multimodal dataset from Amazon Science to test VLM capability to find answers in bulky documents and images.
What's inside
The dataset was created by Amazon Science researchers for a research paper and is designed to evaluate multimodal large language models in long contexts. It contains texts, images, and document pages in English. The dataset focuses on Document QA and Visual Question Answering (VQA) tasks.
What you can do with it
- Benchmarking multimodal LLMs (VLMs) on understanding long multi-page documents
- Evaluating the quality of RAG systems working with scans, PDFs, and complex layouts
- Testing the accuracy of finding facts ("needle in a haystack") in a visual context
- Tasks
- Question answering, Visual question answering, Document question answering
- Languages
- English
- Downloads this month
- 47.6K
- Likes
- 26
How to load it
from datasets import load_dataset
ds = load_dataset("AmazonScience/document-haystack")