Skip to content

Datasets

Hugging Face datasets with plain-language descriptions: what's inside and what models to train on them. 4,788 in the catalog so far, with new ones added every hour.

Data type

For which models

Find a dataset for your goal

Answered by Gemini 4 · the chat is kept only in this tab

Datasetropedia-ai/Ropedia-Xperience-2Top pick

3D scenes and first-person videos for robotics

A large Physical AI dataset with 3D room scans and egocentric recordings in a unified metric system.

  • Robotics
  • Image → 3d
  • Keypoint detection
  • English
9K 6510K–100K rows
Datasetgoogle/multimodal-knowledge-agentTop pick

Google DeepMind benchmark for multimodal agents

10 demonstration tasks from Google DeepMind for evaluating AI agents working with complex graphics and documents.

  • Image text → text
  • Visual question answering
  • Document question answering
  • English
8.8K 103up to 1K rows
DatasetHaoChen2/Gastric-X

Medical CT and Report Dataset

Multi-phase abdominal CT scans with lab panels, reports, organ masks, and tumor masks.

  • Image classification
  • Image segmentation
  • Image → text
  • English
  • Chinese
8.7K 29up to 1K rows🔒 on request
Datasetzzliang/GRIT

GRIT: region-grounded image-text pairs

Large-scale corpus of image-text pairs where caption phrases are grounded to corresponding image regions.

  • Text → image
  • Image → text
  • Object detection
  • English
8.4K 17010M–100M rows
Datasetallenai/CoSyn-400K

Synthetic image-QA pairs

A collection of generated images and question-answer pairs based on executable code from AllenAI.

  • Visual question answering
8.1K 54100K–1M rows
Datasetjmhessel/newyorker_caption_contest

New Yorker humor understanding benchmark

Cartoons and captions from the New Yorker contest for evaluating humor understanding in models.

  • Image → text
  • Multiple choice
  • Text classification
  • English
7.2K 78100K–1M rows
Datasetdatalab-to/omni_parse_bench

Document and table parsing benchmark

A set of 16,288 tests across 2,937 pages for evaluating the accuracy of text, table, and layout recognition.

  • Image → text
7.1K 22100K–1M rows
DatasetPKU-Alignment/align-anything

Multimodal model alignment dataset

Preference score data and text feedback for aligning models across text, images, audio, and tables.

  • Multimodal
7.1K 5010K–100K rows
Datasetmlfoundations/MINT-1T-ArXivTop pick

MINT-1T ArXiv scientific paper multimodal corpus

English interlaced text and image dataset from ArXiv scientific papers for pre-training.

  • Image → text
  • Text generation
  • English
7K 611M–10M rows
Datasetyatin-superintelligence/Edge-Agent-Reasoning-WebSearch-260K

Synthetic reasoning and web search dataset for small models

A 700+ million token corpus for training compact models in step-by-step task analysis and web search.

  • Text generation
  • Question answering
  • Multimodal
  • English
6.7K 55100K–1M rows
Datasetnyu-visionx/Cambrian-Alignment

Multimodal model alignment dataset collection

100K to 1M English image-text question-answer pairs for VLM alignment training.

  • Visual question answering
  • Question answering
  • English
6.6K 39100K–1M rows
Datasetkakaobrain/coyo-700m

747M Image-Text Pairs

A large corpus of alt-text and image pairs from HTML for training multimodal models.

  • Text → image
  • Image → text
  • Zero shot classification
  • English
6.5K 177100M–1B rows
DatasetDelores-Lin/MDPBenchTop pick

Multilingual document parsing benchmark

The first benchmark for evaluating recognition and parsing of real digital and photographed documents across 6 languages.

  • Image → text
  • Chinese
  • English
  • Arabic
  • +9
6.3K 60
Datasetflaviagiammarino/vqa-rad

Radiology image question answering dataset

Question-answer pairs for MedPix medical images for VQA training and testing.

  • Visual question answering
  • English
5.9K 1051K–10K rows
DatasetUkrainianCatholicUniversity/rukopysTop pick

Annotated handwritten Ukrainian texts

The first major open dataset for Ukrainian handwritten text recognition spanning over a century.

  • Object detection
  • Image → text
  • Ukrainian
5.8K 2810K–100K rows
Datasetkurumikz/AnimeOmni

Anime characters with multi-level annotation and descriptions

Over 20 thousand safe-rated anime illustrations with detailed tags, emotions, and descriptions in three languages.

  • Text → image
  • Image → text
  • Image classification
  • English
  • Chinese
  • Japanese
5.5K 1310K–100K rows
Datasetvidore/colpali_train_set

ColPali training set for document retrieval

127K query-image pairs for training visual document retrieval.

  • Document question answering
  • Visual document retrieval
5.3K 93100K–1M rows
Datasetopendatalab/ChartVerse-SFT-1.8MTop pick

Chart and diagram reasoning SFT dataset

1.8M chart QA pairs with Chain-of-Thought annotations for VLM training.

  • Visual question answering
  • Image text → text
  • English
5.1K 1411M–10M rows
DatasetKingdrone-Junjue/EarthVLSet

Satellite imagery, VQA, and Earth surface segmentation

English geospatial dataset for multi-modal analysis and Earth imagery segmentation.

  • Visual question answering
  • Question answering
  • Image segmentation
  • English
4.8K 1210K–100K rows🔒 on request
Datasetnvidia/describe-anything-datasetTop pick

Detailed photo and video object descriptions from NVIDIA

English multimodal dataset for training models to describe specific regions in images and videos in detail.

  • Image → text
  • Video text → text
  • English
4.5K 60100K–1M rows
Datasetjordanparker6/publaynet

Layout markup for scientific article pages

Page images from PubMed Central with layout block annotations for document structure recognition.

  • Image → text
  • English
4.4K 4010K–100K rows
DatasetTeklia/IAM-line

English handwritten text lines for OCR

A classical dataset of individual handwritten text line images in English for OCR training.

  • Image → text
  • English
4.2K 3710K–100K rows