Dataset · HuggingFaceM4
50 image-text datasets for fine-tuning VLMs
HuggingFaceM4/the_cauldron
A collection of 50 training vision-language datasets used for fine-tuning Idefics2.
What's inside
The Cauldron is a large collection of 50 vision-language datasets, training sets only, used when fine-tuning the Idefics2 model. The data includes images and text; size ranges from 1M to 1M examples. Loaded via datasets, for example load_dataset("HuggingFaceM4/the_cauldron", "ai2d").
What you can do with it
- Fine-tune vision-language models like Idefics2
- Evaluate VLMs on various types of image-text tasks
- Build a custom mix for training a multimodal model
- Size
- 1M–10M rows
- Downloads this month
- 127K
- Likes
- 575
Models trained on it
- HuggingFaceTB/SmolVLM2-500M-Video-InstructImage text → text 1.7M
- nicosuter/Qwen3.8-27B-AWQImage text → text 436.5K
- HuggingFaceTB/SmolVLM-500M-InstructImage text → text 258K
- HuggingFaceTB/SmolVLM-256M-InstructImage text → text 229.3K
- HuggingFaceTB/SmolVLM2-2.2B-InstructImage text → text 194.6K
- HuggingFaceTB/SmolVLM2-256M-Video-InstructImage text → text 44.7K
- HuggingFaceM4/Idefics3-8B-Llama3Image text → text 37.6K
- HuggingFaceTB/SmolVLM-InstructImage text → text 16.8K
How to load it
from datasets import load_dataset
ds = load_dataset("HuggingFaceM4/the_cauldron")