Skip to content

Datasets

Hugging Face datasets with plain-language descriptions: what's inside and what models to train on them. 4,788 in the catalog so far, with new ones added every hour.

Data type

For which models

Find a dataset for your goal

Answered by Gemini 4 · the chat is kept only in this tab

Datasetmoonshotai/WorldVQATop pick

Visual World Knowledge Benchmark for MLLMs

Image-based questions to evaluate atomic world knowledge in multimodal models.

  • Visual question answering
  • English
  • Chinese
2.3K 671K–10K rows
Datasetshunk031/STAIR-Captions

Large Japanese image captioning dataset

A large-scale STAIR Captions dataset with 820,310 Japanese image captions for text-to-image and image-to-text generation tasks.

  • Image → text
  • Japanese
2.3K 7100K–1M rows
DatasetByteDance/MTVQA

Multilingual Text-Image VQA (9 Languages)

Image-text question-answer pairs in nine languages, including Russian, for evaluating and training multimodal models.

  • Visual question answering
  • Image → text
  • Multilingual
  • Arabic
  • German
  • +7
2.2K 421K–10K rows
DatasetTIGER-Lab/ViRL39KTop pick

Verifiable visual tasks for VLM RL training

A collection of 38,870 cleaned and verified image-question-answer pairs for training reasoning multimodal models.

  • Question answering
  • Image text → text
  • English
2K 43
DatasetRekaAI/VibeEval

Vibe-Eval benchmark for multimodal chat models

A compact set of challenging vision-language tasks with reference answers in four languages.

  • Image → text
  • Image classification
  • English
  • Polish
  • Chinese
  • +1
1.9K 49up to 1K rows
Datasetvisheratin/laion-coco-nllb

LAION-COCO image captions in 200 languages

A multilingual corpus of image captions translated from English into 200 languages by the NLLB-200 model.

  • Image → text
  • Translation
  • Acehnese
  • acm
  • acq
  • +9
1.7K 46100K–1M rows
Datasetproject-oceania/planktonzilla-17MTop pick

17 million labeled plankton images

Largest combined dataset of plankton photographs with taxonomic labeling.

  • Image → text
  • Image classification
  • Image text → text
  • English
1.7K 810M–100M rows
DatasetHelenPeng/ExeWorldBenchTop pick

Executable 3D world code generation benchmark

An English multimodal benchmark for evaluating whether models can write working 3D scene code.

  • Text generation
  • Image text → text
  • English
1.7K 411K–10K rows
Datasetvidore/docvqa_test_subsampled

DocVQA test subset for document retrieval

A subset of 500 English question-answer pairs for document scans to evaluate multimodal retrieval and VQA.

  • Document question answering
  • Visual document retrieval
  • English
1.6K 7up to 1K rows
Datasetgigant/oldbookillustrations

Scans of vintage book illustrations with descriptions

A collection of 4,172 scanned engravings and drawings from old books with text captions for image generation and analysis.

  • Text → image
  • Image → text
  • Image → image
  • English
  • French
  • German
1.6K 411K–10K rows
Datasetallenai/pixmo-cap

Long and detailed image descriptions for VLMs

Hundreds of thousands of images with detailed texts averaging 200 words and original speech transcripts.

  • Image → text
1.5K 49100K–1M rows
Datasetopenbmb/RLHF-V-Dataset

Multimodal preferences for RLHF

A small set of multimodal feedback for aligning vision-language models.

  • Text generation
  • Visual question answering
  • English
1.5K 871K–10K rows
Datasetdinhanhx/crossmodal-3600

Multilingual image description dataset

Set of 3,600 global images with reference descriptions in 36 languages.

  • Image → text
  • Arabic
  • Bangla
  • Czech
  • +9
1.4K 71K–10K rows
DatasetCATMuS/modern

CATMuS Modern handwritten text corpus

An image and text dataset for Handwritten Text Recognition (HTR) tasks in multiple European languages.

  • Image → text
  • French
  • German
  • English
  • +4
1.4K 5100K–1M rows
DatasetBUAADreamer/llava-en-zh-300k

English-Chinese bilingual multimodal instructions

300k image-question examples for training vision-language models in LLaMA Factory.

  • Text generation
  • Visual question answering
  • English
  • Chinese
1.3K 36100K–1M rows
DatasetML2/navocr-testsets

Test sets for navigation text recognition

1,000 to 10,000 labeled photos of Korean and English signs to evaluate mobile robot OCR systems.

  • Object detection
  • Image → text
  • Korean
  • English
1.3K 21K–10K rows
Datasethuaweilin/ViRA-Train

ViRA training data for visual agents

A training dataset for the ViRA model designed for ultra-high-resolution tasks.

  • Visual question answering
  • English
1.3K 21K–10K rows
Datasetraidium/RadImageNet-VQATop pick

CT/MRI dataset for radiological VQA

750K CT/MRI images and 7.5M generated VQA examples with expert annotations for training and benchmarking.

  • Visual question answering
  • English
1.3K 971M–10M rows🔒 on request
Datasetantins-labs/VisSearch-Bench-v1

VisSearch-Bench Visual Search Benchmark

204 English questions with 190 images for evaluating long visual search in multimodal agents.

  • Image text → text
  • English
1.2K 7up to 1K rows
DatasetLouisChen15/ConstructionSite

Annotated construction sites

10,013 construction site images with annotations for evaluating scene understanding in vision-language models.

  • Image → text
  • Image feature extraction
  • English
1.2K 8410K–100K rows🔒 on request
Datasetjingyaogong/minimind-v_dataset

Bilingual image-text pairs for pre-training and SFT

A large Chinese-English corpus based on ALLaVA-4V for training and fine-tuning vision-language models.

  • Visual question answering
  • Chinese
  • English
1.2K 39up to 1K rows
Datasetterryoo/TableVQA-Bench

Table and image visual question answering benchmark

English dataset for evaluating models in Visual Question Answering and table comprehension tasks.

  • Table question answering
  • Visual question answering
  • English
1.1K 351K–10K rows