Dataset · mlfoundations
Multimodal PDF and web page corpus (MINT-1T part)
mlfoundations/MINT-1T-PDF-CC-2023-14
A slice of the ultra-large MINT-1T dataset of 1M to 10M records with interleaved English text and images.
What's inside
The dataset contains pairs and sequences of images and English texts extracted from web pages and PDF documents. This repository is a subset ranging from 1M to 10M records, part of the giant open MINT-1T corpus (1 trillion tokens and 3.4 billion images). The collection was conducted by a team from the University of Washington.
What you can do with it
- Pre-training multimodal models (VLM) on text-image pairs
- Training models to understand complex PDF documents and layout
- Image-to-text generation
- Tasks
- Image → text, Text generation
- Languages
- English
- Size
- 1M–10M rows
- Downloads this month
- 22.9K
- Likes
- 6
How to load it
from datasets import load_dataset
ds = load_dataset("mlfoundations/MINT-1T-PDF-CC-2023-14")