Skip to content

Dataset · mlfoundations

Multimodal PDF and web page corpus (MINT-1T part)

mlfoundations/MINT-1T-PDF-CC-2023-14

A slice of the ultra-large MINT-1T dataset of 1M to 10M records with interleaved English text and images.

What's inside

The dataset contains pairs and sequences of images and English texts extracted from web pages and PDF documents. This repository is a subset ranging from 1M to 10M records, part of the giant open MINT-1T corpus (1 trillion tokens and 3.4 billion images). The collection was conducted by a team from the University of Washington.

What you can do with it

  • Pre-training multimodal models (VLM) on text-image pairs
  • Training models to understand complex PDF documents and layout
  • Image-to-text generation
Tasks
Image → text, Text generation
Languages
English
Size
1M–10M rows
Downloads this month
22.9K
Likes
6

How to load it

from datasets import load_dataset

ds = load_dataset("mlfoundations/MINT-1T-PDF-CC-2023-14")