Skip to content

Datasets

Hugging Face datasets with plain-language descriptions: what's inside and what models to train on them. 4,788 in the catalog so far, with new ones added every hour.

Data type

For which models

Find a dataset for your goal

Answered by Gemini 4 · the chat is kept only in this tab

Datasetnyu-visionx/VSI-Bench

Video visual-spatial VQA benchmark

Video question-answering dataset for testing visual-spatial understanding in multimodal models.

  • Visual question answering
  • English
28.8K 7810K–100K rows
DatasetVillanovaAI/Multi-CoSyn-400K

Multilingual synthetic document VQA dataset

100K to 1M question-answer pairs with reasoning over text-dense images across 5 European languages.

  • Visual question answering
  • Italian
  • English
  • German
  • +2
28.7K 0100K–1M rows🔒 on request
DatasetCASIA-IVA-Lab/VisReason

VisReason visual reasoning benchmark

A set of 1,505 visual questions to test multimodal model image reasoning capabilities.

  • Visual question answering
28.6K 01K–10K rows
Datasetalibaba-multimodal-industrial-ai/IndustryBench-MIPU

Industrial product feature extraction benchmark

Image and Chinese text dataset for evaluating MLLMs in multi-image attribute extraction.

  • Image → text
  • Visual question answering
  • Chinese
27.6K 810K–100K rows
DatasetPawChrona/PawChrona-MMQA

Pet Health and Behavior Multimodal Q&A Dataset

49K English and Chinese questions about cats and dogs with images, audio, video, and text.

  • Visual question answering
  • Question answering
  • English
  • Chinese
27.4K 11100K–1M rows🔒 on request
Datasethashmortar/spreadsheet-bench-v2-modified

Table, PDF, and document question answering

A set of 1,060 questions and reference answers in English across 171 business documents in various formats.

  • Question answering
  • Table question answering
  • Document question answering
  • English
27.2K 21K–10K rows
DatasetBoKelvin/SLAKE

SLAKE Medical Visual Question Answering Dataset

Medical image dataset with questions, answers, and semantic annotations in English and Chinese.

  • Visual question answering
  • English
  • Chinese
26.8K 6510K–100K rows
DatasetHuggingFaceM4/DocmatixTop pick

Documents and questions for Document VQA

Massive English dataset for Document VQA used in Idefics3 fine-tuning.

  • Visual question answering
  • English
26.8K 3191M–10M rows
DatasetRevolutionCrossroads/loc_chronicling_america_1770-1810

OCR Archive of 1770-1810 American Newspapers

Historical US newspaper pages with images, original OCR, and neural text recognition.

  • Image → text
  • Text retrieval
  • Text classification
  • English
26.6K 810K–100K rows
DatasetMC7ever/hf-training-corpus

Multimodal HF Corpus of 92K Datasets

Large-scale extraction of texts, images, audio, and video from open datasets in a unified parquet format.

  • Text generation
  • Image → text
  • Automatic speech recognition
  • English
26.5K 1100K–1M rows
DatasetForithmus/MR-RATE-coreg

Aligned volumetric MRI scans for medical models

10k to 100k coregistered MRI volumes brought to a single coordinate system.

  • Image → text
  • Text → image
  • Image classification
  • English
26.4K 510K–100K rows🔒 on request
DatasetManiTwin/ManiTwin-100K

3D digital object twins for robotics

3D meshes with physical properties, grasp points, and descriptions for robot training.

  • Robotics
  • Image → 3d
  • Visual question answering
26.2K 13100K–1M rows
DatasetLin-Chen/MMStar

Data-Leakage-Free Multimodal Benchmark

Curated English visual-text tasks for fair evaluation of large multimodal models.

  • Multiple choice
  • Question answering
  • Visual question answering
  • English
26.2K 551K–10K rows
Datasetmlfoundations/MINT-1T-PDF-CC-2023-50Top pick

MINT-1T: interleaved text and images from PDF and the web

Part of a massive 1 trillion token multimodal dataset for pre-training models in document and page understanding.

  • Image → text
  • Text generation
  • English
25.8K 141M–10M rows
DatasetDongping-Li/EMMOE-100

Robotics, RL, and VQA training dataset

An English dataset of 10K–100K examples featuring scene descriptions, trajectories, and VQA tasks for embodied AI.

  • Visual question answering
  • Reinforcement learning
  • Robotics
  • English
25.7K 110K–100K rows
Datasetayushsdev/qvhighlights-videos

Complete QVHighlights Benchmark Video Set

25,124 English MP4 video clips for text-to-moment retrieval tasks.

  • Video classification
  • Video text → text
  • English
25.4K 010K–100K rows
Datasetdocling-project/DeskForge-1MTop pick

1.21m labeled desktop screenshots with clicks

A corpus of desktop screenshots annotated with UI elements and click transitions for training GUI agents.

  • Image → text
  • Object detection
  • English
25.3K 521M–10M rows
DatasetOneThink/OneThinker-train-dataTop pick

OneThinker-600k multimodal dataset

Large-scale multimodal data corpus for training a universal reasoning model.

  • Image text → text
  • Video text → text
  • Object detection
  • English
24.3K 1410K–100K rows
Datasetlaurenssam/REST

REST rendering equivalence test for VLM

Dataset for testing the robustness of Vision-Language Models to text or image prompt inputs.

  • Visual question answering
  • Question answering
  • Image text → text
  • English
24K 010K–100K rows
Datasetpixparse/pdfa-eng-wds

PDF pages for vision-language models

English PDF pages from SafeDocs with annotations for training and evaluating OCR and vision-language models.

  • Image → text
  • English
23.8K 16810M–100M rows
Dataset1fanj/Chartographer

Visual chart analysis benchmark

English chart-question pairs to evaluate whether VLMs actually analyze images instead of guessing.

  • Visual question answering
  • Question answering
  • English
23.7K 31K–10K rows