Skip to content

Datasets

Hugging Face datasets with plain-language descriptions: what's inside and what models to train on them. 4,788 in the catalog so far, with new ones added every hour.

Data type

For which models

Find a dataset for your goal

Answered by Gemini 4 · the chat is kept only in this tab

DatasetDongping-Li/EMMOE-100

Robotics, RL, and VQA training dataset

An English dataset of 10K–100K examples featuring scene descriptions, trajectories, and VQA tasks for embodied AI.

  • Visual question answering
  • Reinforcement learning
  • Robotics
  • English
25.7K 110K–100K rows
DatasetThinkingHub/PPTop pick

SteelBench VLM industrial safety benchmark

A dataset of annotated CCTV clips for evaluating VLMs in industrial safety.

  • Video classification
  • Image classification
  • Visual question answering
  • English
25.4K 01K–10K rows
Dataset1fanj/Chartographer

Visual chart analysis benchmark

English chart-question pairs to evaluate whether VLMs actually analyze images instead of guessing.

  • Visual question answering
  • Question answering
  • English
23.9K 31K–10K rows
DatasetRevolutionCrossroads/nara_revolutionary_war_pension_files_PDFs

NARA Revolutionary War pension files archive

10k to 100k digitized multi-page National Archives documents in English.

  • Image → text
  • Text retrieval
  • Text classification
  • English
23.6K 010K–100K rows
DatasetUCSC-VLAA/Recap-DataComp-1BTop pick

1B image-text pairs with re-captions

Large DataComp-1B image-text corpus with improved captions from LLaVA-1.5-LLaMA3-8B.

  • Zero shot classification
  • Text retrieval
  • Image → text
23.5K 2131B–10B rows
Datasetckchaos/ChartDiff

ChartDiff chart pair comparison benchmark

Large-scale benchmark for comparative summarization of chart pairs by language models.

  • Summarization
  • Image text → text
  • Image → text
23.1K 01K–10K rows
DatasetOmniEdu/Omni-Edu

Omni-Edu Multimodal Dataset

Instructional dataset for educational tasks and multimodal learning.

  • Text generation
  • Question answering
  • Image text → text
  • English
  • Chinese
23K 010K–100K rows
Datasetmlfoundations/MINT-1T-PDF-CC-2023-14

Multimodal PDF and web page corpus (MINT-1T part)

A slice of the ultra-large MINT-1T dataset of 1M to 10M records with interleaved English text and images.

  • Image → text
  • Text generation
  • English
22.9K 61M–10M rows
Datasethashmortar/spreadsheet-bench-v2-modified

Table, PDF, and document question answering

A set of 1,060 questions and reference answers in English across 171 business documents in various formats.

  • Question answering
  • Table question answering
  • Document question answering
  • English
22.8K 21K–10K rows
Datasetflaviagiammarino/path-vqa

PathVQA: pathology visual question answering

Question-answer pairs on pathology images for training and evaluating medical VQA systems.

  • Visual question answering
  • English
22.5K 7710K–100K rows
DatasetMCG-NJU/VideoChat3-LV116k

Long video analysis instructions (VideoChat3)

An English video-text instruction dataset for training models to understand long videos.

  • Video text → text
  • English
22.2K 151K–10K rows
Datasetmorgan/CADExpert_fixed

CADExpert dataset of CAD tasks

14,935 unique CAD tasks with images and three expert answers each.

  • Image → text
  • English
  • Chinese
21.9K 110K–100K rows
DatasetHothan/OlympiadBench

Multimodal benchmark of Olympiad scientific problems

Complex Olympiad-level problems with images in English and Chinese for evaluating model scientific reasoning.

  • Question answering
  • Visual question answering
  • Chinese
  • English
21.8K 521K–10K rows
Datasetimageomics/VLM4Bio

VLM4Bio dataset for ornithology

Metadata and scripts for downloading bird images for visual question-answering tasks.

  • Visual question answering
  • Zero shot image classification
  • Zero shot object detection
  • English
21.6K 110K–100K rows
DatasetSimulaMet/Kvasir-VQA-x1

Medical dataset Kvasir-VQA-x1 for endoscopy

Multimodal dataset of 159,549 question-answer pairs for benchmarking in gastrointestinal endoscopy.

  • Visual question answering
  • Image text → text
  • English
21.6K 18100K–1M rows
Datasetyingss/mixlora-eval-data

MixLoRA multimodal eval dataset

Multimodal evaluation dataset and MME benchmark from the MixLoRA paper (ACL 2024).

  • Visual question answering
  • Image classification
  • English
21.6K 01K–10K rows
Datasetmlfoundations/MINT-1T-PDF-CC-2024-10Top pick

Multimodal PDF corpus for VLM pre-training

Part of the giant open MINT-1T dataset featuring interleaved English text and PDF images.

  • Image → text
  • Text generation
  • English
21.5K 51M–10M rows
Datasetgksriharsha/chitralekha

Large Telugu OCR dataset

Tens of millions of text image and transcript pairs for training Telugu optical character recognition systems.

  • Image → text
  • Telugu
21.5K 1110M–100M rows
Datasetjinlong666/IndustryBench-MIPU

Industry Document Specification Extraction Benchmark

A collection of Chinese technical tables and blueprint images for evaluating Multimodal Large Language Models (MLLMs).

  • Image → text
  • Visual question answering
  • Chinese
21.4K 110K–100K rows
DatasetSinerChen/SceneBench

Long video scene understanding benchmark

A collection of videos and questions in English for evaluating scene event understanding by multimodal LLMs.

  • Visual question answering
  • English
21.2K 3up to 1K rows
DatasetMCG-NJU/TimeLens2-93KTop pick

Temporal grounding in long videos (93k pairs)

Large dataset for text-based event retrieval in video clips up to 100 minutes long.

  • Video text → text
  • English
  • Chinese
  • Japanese
  • +1
21.2K 1310K–100K rows
DatasetVincent-HKUSTGZ/FragFake

AI Image Editing Detection Dataset via VQA

Image-text pairs for training multimodal models to detect image editing artifacts.

  • Visual question answering
  • English
20.9K 110K–100K rows
Datasetmlfoundations/MINT-1T-PDF-CC-2023-23Top pick

MINT-1T multimodal corpus chunk (PDF and web data)

Part of a giant open dataset with interleaved texts and images from PDFs and the web.

  • Image → text
  • Text generation
  • English
20.6K 101M–10M rows