Skip to content

Dataset · AmazonScience

Long Document Analysis Benchmark for Visual LLMs

AmazonScience/document-haystack

Multimodal dataset from Amazon Science to test VLM capability to find answers in bulky documents and images.

What's inside

The dataset was created by Amazon Science researchers for a research paper and is designed to evaluate multimodal large language models in long contexts. It contains texts, images, and document pages in English. The dataset focuses on Document QA and Visual Question Answering (VQA) tasks.

What you can do with it

  • Benchmarking multimodal LLMs (VLMs) on understanding long multi-page documents
  • Evaluating the quality of RAG systems working with scans, PDFs, and complex layouts
  • Testing the accuracy of finding facts ("needle in a haystack") in a visual context
Tasks
Question answering, Visual question answering, Document question answering
Languages
English
Downloads this month
47.6K
Likes
26

How to load it

from datasets import load_dataset

ds = load_dataset("AmazonScience/document-haystack")