Dataset · neulab
MultiUI multimodal web interface dataset (7.3M)
neulab/MultiUI
7.3 million multimodal examples from 1 million websites for training web agents and VLM models.
What's inside
Large English research paper corpus containing 7.3 million text-image pairs collected from 1 million web pages with diverse layouts. Data is annotated for question-answering systems and visual interface understanding (VQA) tasks. The authors note an increase in web agent accuracy of up to 19.1% and improved results on VisualWebBench up to 48%.
What you can do with it
- Training multimodal models (VLM) to recognize elements and text on website screenshots
- Training autonomous browser AI agents for navigation and action execution in web interfaces
- Tasks
- Question answering, Visual question answering
- Languages
- English
- Size
- 1M–10M rows
- Downloads this month
- 847
- Likes
- 53
- Access
- on request on Hugging Face
Models trained on it
How to load it
from datasets import load_dataset
ds = load_dataset("neulab/MultiUI")