Dataset · mlfoundations
Chunk of the massive MINT-1T multimodal corpus (PDF)
mlfoundations/MINT-1T-PDF-CC-2023-40
Part of the open MINT-1T dataset with interleaved text and images from English PDFs and research papers.
What's inside
A fragment of the large-scale open MINT-1T corpus created by University of Washington researchers for multimodal pre-training. The overall project combines 1 trillion text tokens and 3.4 billion images, including data from previously unused PDF documents and ArXiv articles; this repository contains 100 billion to 1 trillion English records. File format is not specified in the description.
What you can do with it
- Pre-training large multimodal models (VLMs) on text-image pairs
- Training systems to recognize and understand complex documents, PDFs, and research papers
- Tasks
- Image → text, Text generation
- Languages
- English
- Size
- 100B–1T rows
- Downloads this month
- 29.6K
- Likes
- 9
How to load it
from datasets import load_dataset
ds = load_dataset("mlfoundations/MINT-1T-PDF-CC-2023-40")