Skip to content

Dataset · mlfoundations

DataComp XLarge giant pool metadata

mlfoundations/datacomp_xlarge

Image links and text descriptions (10 to 100 billion records) for large-scale model pre-training.

What's inside

The repository stores metadata and pairs of image URL links and texts for the ultra-large DataComp XLarge pool, distributed under the CC-BY-4.0 license. The image files themselves are not included in the archive and are downloaded separately via links subject to their own copyrights.

What you can do with it

  • Large-scale pre-training of CLIP-level multimodal models
  • Researching filtering and selection methods for ultra-large web datasets
Size
10B–100B rows
Downloads this month
19.9K
Likes
22

How to load it

from datasets import load_dataset

ds = load_dataset("mlfoundations/datacomp_xlarge")