Skip to content

Datasets

Hugging Face datasets with plain-language descriptions: what's inside and what models to train on them. 4,788 in the catalog so far, with new ones added every hour.

Data type

For which models

Find a dataset for your goal

Answered by Gemini 4 · the chat is kept only in this tab

DatasetAI4Math/MathVerse

MathVerse: visual math problems

Visual math problem benchmark for multimodal LLMs.

  • Multiple choice
  • Question answering
  • Visual question answering
  • English
9.3K 731K–10K rows
Datasetyuhos16/SkinCaReTop pick

SkinCaRe multimodal dermatology dataset

7,041 dermatology cases with medical descriptions and Chain-of-Thought reasoning.

  • Image → text
  • Chinese
  • English
9.1K 16up to 1K rows🔒 on request
DatasetMMInstruction/M3IT

Multimodal instructions

Multimodal instructions in English and Chinese for training vision-language models; 1 to 10 million examples.

  • Image → text
  • Image classification
  • English
  • Chinese
9.1K 1361M–10M rows
Datasetropedia-ai/Ropedia-Xperience-2Top pick

3D scenes and first-person videos for robotics

A large Physical AI dataset with 3D room scans and egocentric recordings in a unified metric system.

  • Robotics
  • Image → 3d
  • Keypoint detection
  • English
9K 6510K–100K rows
Datasetmoonshotai/PerceptionBench

Multimodal LLM visual perception benchmark

Test suite from Moonshot AI for evaluating basic visual perception in multimodal models.

  • Visual question answering
  • English
8.8K 541K–10K rows
Datasetgoogle/multimodal-knowledge-agentTop pick

Google DeepMind benchmark for multimodal agents

10 demonstration tasks from Google DeepMind for evaluating AI agents working with complex graphics and documents.

  • Image text → text
  • Visual question answering
  • Document question answering
  • English
8.8K 103up to 1K rows
DatasetHaoChen2/Gastric-X

Medical CT and Report Dataset

Multi-phase abdominal CT scans with lab panels, reports, organ masks, and tumor masks.

  • Image classification
  • Image segmentation
  • Image → text
  • English
  • Chinese
8.7K 29up to 1K rows🔒 on request
Datasetzzliang/GRIT

GRIT: region-grounded image-text pairs

Large-scale corpus of image-text pairs where caption phrases are grounded to corresponding image regions.

  • Text → image
  • Image → text
  • Object detection
  • English
8.4K 17010M–100M rows
Datasetgajeshladhar/core-fiveTop pick

Earth Observation multimodal satellite data

A large geospatial dataset with temporally and spatially aligned imagery from five satellite sensors.

  • Image segmentation
  • Image classification
  • Object detection
  • English
8.2K 4010M–100M rows
Datasetallenai/CoSyn-400K

Synthetic image-QA pairs

A collection of generated images and question-answer pairs based on executable code from AllenAI.

  • Visual question answering
8.1K 54100K–1M rows
DatasetOpenMOSS-Team/OmniAction-LIBERO

Robotic manipulation in an omnimodal context

Data for proactive robotic manipulation with audio and multimodal context.

  • Robotics
  • Multimodal
  • Audio → audio
  • English
7.5K 70
Datasetnvidia/Nemotron-Image-Training-v3

Multimodal data for vision-language models

A large collection of image-centric data for training and evaluating vision-language models; subsets in JSONL with descriptions.

  • Visual question answering
  • Image text → text
7.4K 871M–10M rows
Datasetjackyhate/text-to-image-2M

2M text-image pairs for T2I

A high-quality and diverse dataset of approximately 2 million text-image pairs for text-to-image training.

  • Text → image
  • Image → text
  • Image classification
  • English
7.2K 1981M–10M rows
Datasetjmhessel/newyorker_caption_contest

New Yorker humor understanding benchmark

Cartoons and captions from the New Yorker contest for evaluating humor understanding in models.

  • Image → text
  • Multiple choice
  • Text classification
  • English
7.2K 78100K–1M rows
Datasetdatalab-to/omni_parse_bench

Document and table parsing benchmark

A set of 16,288 tests across 2,937 pages for evaluating the accuracy of text, table, and layout recognition.

  • Image → text
7.1K 22100K–1M rows
DatasetPKU-Alignment/align-anything

Multimodal model alignment dataset

Preference score data and text feedback for aligning models across text, images, audio, and tables.

  • Multimodal
7.1K 5010K–100K rows
Datasetmlfoundations/MINT-1T-ArXivTop pick

MINT-1T ArXiv scientific paper multimodal corpus

English interlaced text and image dataset from ArXiv scientific papers for pre-training.

  • Image → text
  • Text generation
  • English
7K 611M–10M rows
DatasetScienceOne-AI/S1-MMAlign

Scientific multimodal image-text pairs

24.17M image-text pairs from 3.01M open-access English scientific papers.

  • Image → text
  • Visual question answering
  • Embeddings
  • English
6.9K 10710M–100M rows
Datasetyatin-superintelligence/Edge-Agent-Reasoning-WebSearch-260K

Synthetic reasoning and web search dataset for small models

A 700+ million token corpus for training compact models in step-by-step task analysis and web search.

  • Text generation
  • Question answering
  • Multimodal
  • English
6.7K 55100K–1M rows
Datasetnyu-visionx/Cambrian-Alignment

Multimodal model alignment dataset collection

100K to 1M English image-text question-answer pairs for VLM alignment training.

  • Visual question answering
  • Question answering
  • English
6.6K 39100K–1M rows
Datasetjamescalam/youtube-transcriptions

Technical YouTube Tutorial Transcriptions

Short text snippets from English video tutorials with timestamps for search and RAG.

  • Question answering
  • Text retrieval
  • Visual question answering
  • English
6.5K 44100K–1M rows