Skip to content

Datasets

Hugging Face datasets with plain-language descriptions: what's inside and what models to train on them. 4,788 in the catalog so far, with new ones added every hour.

Data type

For which models

Find a dataset for your goal

Answered by Gemini 4 · the chat is kept only in this tab

Datasetmaritaca-ai/enem

Brazilian ENEM 2022-2024 exam questions

Multiple-choice tasks from the Brazilian university entrance exam for evaluating LLMs and VLMs.

  • Visual question answering
  • Multiple choice
  • Portuguese
3.5K 18up to 1K rows
DatasetXkev/LLaVA-CoT-100k

100k VQA examples with step-by-step reasoning

VQA data with structured reasoning annotations for training Vision-Language Models.

  • Visual question answering
  • Image text → text
  • English
3.5K 10910K–100K rows
Datasetzhihengli-casia/puredocbench

Three-mode document recognition benchmark

From 1 to 10k document pages for comparing OCR and vision-language models across three tracks.

  • Image → text
  • Document question answering
3.4K 61K–10K rows
DatasetProGamerGov/synthetic-dataset-1m-dalle3-high-quality-captions

1M AI Images with High-Quality Captions

Over a million AI-generated images with detailed English captions for text-to-image and image-to-text tasks.

  • Text → image
  • Image classification
  • Image → text
  • English
3.4K 1621M–10M rows
DatasetUniParser/OmniScienceTop pick

Scientific images with text descriptions

A large multimodal dataset for understanding scientific images: images and text for image-to-text tasks.

  • Image → text
3.3K 1301M–10M rows🔒 on request
DatasetFBK-MT/MCIFTop pick

Multilingual multimodal benchmark on scientific reports

Human-annotated instruction dataset based on scientific reports: speech, video, and text across four languages to evaluate MLLMs.

  • Automatic speech recognition
  • Question answering
  • Summarization
  • English
  • German
  • Italian
  • +1
3.2K 711K–10K rows
Datasetvisual-layer/imagenet-1k-vl-enrichedTop pick

Enriched ImageNet-1K with descriptions and object boxes

An augmented version of ImageNet-1K with over a million records containing text descriptions, bounding boxes, and label error notes.

  • Object detection
  • Image classification
  • Text → image
  • English
3.1K 411M–10M rows
DatasetTencentARC/Plot2Code

Chart-to-code generation benchmark for MLLMs

Chart images and English texts to evaluate how multimodal models write code.

  • Text generation
  • Text → image
  • Image → text
  • English
3.1K 36up to 1K rows
Datasetkdexd/red_caps

12M image-caption pairs from Reddit

Large English corpus of 12M images and captions from 350 subreddits for image-to-text tasks.

  • Image → text
  • English
3.1K 6010M–100M rows
DatasetShareGPTVideo/train_video_and_instruction

Video frames and instructions for multimodal models

Dataset of 900k video frames with English questions and instructions for pretraining and SFT of video-language models.

  • Question answering
  • Video text → text
  • English
3.1K 35
DatasetBAAI/DenseFusion-1MTop pick

1M Dataset with Detailed Image Descriptions

A large multimodal set of 1 million to 10 million image-text pairs in English.

  • Question answering
  • Visual question answering
  • English
3K 441M–10M rows
Datasetranjaykrishna/visual_genome

Images with dense object annotations

108K images with objects, attributes, and pairwise relationships for vision and language tasks.

  • Image → text
  • Object detection
  • Visual question answering
  • English
2.9K 86100K–1M rows
DatasetHuggingFaceM4/DoclingMatix

Multimodal document recognition dataset

From 1 to 10 million document image-text pairs in English for training VLM models.

  • Visual question answering
  • Image text → text
  • English
2.9K 581M–10M rows
Datasetaipracticecafe/curated-danbooru-2026

Curated Danbooru anime illustrations with tags

Curated dataset of 330 thousand high-quality anime arts with tags and prompts for training image generators.

  • Text → image
  • Image → text
  • English
2.9K 17100K–1M rows
DatasetMMInstruction/ArxivQA

Multimodal QA on arXiv figures and charts

Over 100k multiple-choice questions based on scientific images from arXiv articles.

  • Image → text
  • English
2.8K 38100K–1M rows
Datasetopenbmb/RLAIF-V-Dataset

RLAIF-V data for trustworthy VLMs

English images and text for training and validating trustworthy multimodal models via RLAIF.

  • Image text → text
  • Visual question answering
  • Multimodal
  • English
2.6K 22710K–100K rows