Skip to content

Datasets

Hugging Face datasets with plain-language descriptions: what's inside and what models to train on them. 4,788 in the catalog so far, with new ones added every hour.

Data type

For which models

Find a dataset for your goal

Answered by Gemini 4 · the chat is kept only in this tab

Datasetbones-studio/seedTop pick

Annotated human motions for robots

142,220 human motion animations with text descriptions and annotations for humanoid robotics.

  • Robotics
  • Text → video
  • Video text → text
  • English
17.1K 515100K–1M rows🔒 on request
Datasetpixparse/idl-wds

UCSF documents with OCR annotations

Millions of document pages with PDF, TIFF images, and OCR annotations for recognition training.

  • Image → text
16.9K 20710M–100M rows
Datasetgoogle/RSRCCTop pick

Google Remote Sensing Change Captioning Benchmark

A Google Research dataset with multi-temporal aerospace image pairs and questions about terrain changes.

  • Visual question answering
  • Image text → text
  • Multiple choice
  • English
16.2K 41100K–1M rows
DatasetMathLLMs/MathVision

MATH-Vision benchmark for multimodal mathematics

Visual math problems with images and text for evaluating VLMs.

  • Question answering
  • Multiple choice
  • Visual question answering
  • English
15.7K 1841K–10K rows
DatasetMAmmoTH-VL/MAmmoTH-VL-Instruct-12M

12M visual instructions for multimodal models

Image-instruction pairs for training visual assistants in English.

  • Visual question answering
  • Question answering
  • English
15.4K 6710M–100M rows
DatasetQwen/Qwen-Image-BenchTop pick

Qwen Image Generation Benchmark

An English and Chinese image-text dataset for deep evaluation of Text-to-Image models.

  • Image → text
  • English
  • Chinese
15K 531K–10K rows
DatasetOpen-Bee/Honey-Data-15M

Bee multimodal image-text corpus

15M English image-text pairs for training and evaluating open multimodal models.

  • Image text → text
  • English
15K 12110M–100M rows
Datasetxlangai/AgentNetTop pick

OpenCUA desktop agent trajectories

22.6K human-annotated tasks for computer-use agents on Windows, macOS, and Ubuntu.

  • Image text → text
  • English
14.8K 99
DatasetShareGPT4Video/ShareGPT4Video

ShareGPT4Video: video captions

4.8M multimodal GPT-4-Vision video descriptions for LVLM and text-to-video models.

  • Visual question answering
  • Question answering
  • English
14K 21610K–100K rows
DatasetBAAI/Infinity-MM

Tens of millions of multimodal instructions

A large multimodal instruction dataset from BAAI: images with QA pairs in English and Chinese.

  • Image → text
  • English
  • Chinese
13.5K 12010M–100M rows🔒 on request
DatasetShadenA/MathNet

Multilingual math problems with images

Problems from countries and competitions with images and text for QA and solution generation.

  • Question answering
  • Text generation
  • Image → text
  • English
  • Portuguese
  • Spanish
  • +9
13.4K 10510K–100K rows
DatasetMME-Benchmarks/Video-MME-v2

Video-MME v2 Video Understanding Benchmark

Updated benchmark with English videos and annotations for comprehensive evaluation of video models.

  • Video text → text
  • English
13.1K 521K–10K rows
Datasetliuhaotian/LLaVA-Instruct-150K

Multimodal instructions on 150k examples

150k image-instruction pairs from GPT-4 for visual instruction tuning and VQA.

  • Visual question answering
  • Question answering
  • English
12.7K 650100K–1M rows
Datasetwikimedia/wit_base

Multilingual image-text pairs from Wikipedia

Image and text connections from Wikipedia across many languages for training and evaluating multimodal retrieval.

  • Image → text
  • Text retrieval
  • Afrikaans
  • Aragonese
  • Arabic
  • +9
12.5K 77100K–1M rows
Datasetnyu-visionx/CV-Bench

CV-Bench visual benchmark for multimodal LLMs

1K to 10K test examples with images and questions to evaluate vision capabilities in multimodal models.

  • Visual question answering
  • English
12.2K 491K–10K rows
Datasetnvidia/Nemotron-VLM-Dataset-v2Top pick

Nemotron VLM dataset v2

Nearly triple the high-quality samples of v1 for training VLM on image and video QA.

  • Visual question answering
  • Image text → text
  • Video text → text
11.6K 991M–10M rows
Datasetinternlm/WildClawBench

WildClawBench agent benchmark

60 practical tasks for evaluating AI agents in a live OpenClaw environment.

  • Visual question answering
  • Image text → text
  • Question answering
  • English
  • Chinese
11.5K 73up to 1K rows
Datasetomegalabsinc/omega-multimodal

Multimodal video dataset from the Bittensor network

A massive video corpus with text descriptions and tabular metadata from the decentralized OMEGA Labs network.

  • Video text → text
  • Video classification
  • Image classification
11.3K 61
DatasetHaoChen2/Gastric-X

Medical CT and Report Dataset

Multi-phase abdominal CT scans with lab panels, reports, organ masks, and tumor masks.

  • Image classification
  • Image segmentation
  • Image → text
  • English
  • Chinese
11.1K 29up to 1K rows🔒 on request
Datasethuggan/wikiart

WikiArt paintings with labels

81,444 art images with artist, genre, and style labels.

  • Image classification
  • Text → image
  • Image → text
10K 25010K–100K rows