Skip to content

Dataset · HuggingFaceM4

50 image-text datasets for fine-tuning VLMs

HuggingFaceM4/the_cauldron

A collection of 50 training vision-language datasets used for fine-tuning Idefics2.

What's inside

The Cauldron is a large collection of 50 vision-language datasets, training sets only, used when fine-tuning the Idefics2 model. The data includes images and text; size ranges from 1M to 1M examples. Loaded via datasets, for example load_dataset("HuggingFaceM4/the_cauldron", "ai2d").

What you can do with it

  • Fine-tune vision-language models like Idefics2
  • Evaluate VLMs on various types of image-text tasks
  • Build a custom mix for training a multimodal model
Size
1M–10M rows
Downloads this month
127K
Likes
575

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("HuggingFaceM4/the_cauldron")