Skip to content

Datasets

Hugging Face datasets with plain-language descriptions: what's inside and what models to train on them. 4,788 in the catalog so far, with new ones added every hour.

Data type

For which models

Find a dataset for your goal

Answered by Gemini 4 · the chat is kept only in this tab

Datasetjinlong666/IndustryBench-MIPU

Industry Document Specification Extraction Benchmark

A collection of Chinese technical tables and blueprint images for evaluating Multimodal Large Language Models (MLLMs).

  • Image → text
  • Visual question answering
  • Chinese
21.4K 110K–100K rows
DatasetSinerChen/SceneBench

Long video scene understanding benchmark

A collection of videos and questions in English for evaluating scene event understanding by multimodal LLMs.

  • Visual question answering
  • English
21.2K 3up to 1K rows
DatasetMCG-NJU/TimeLens2-93KTop pick

Temporal grounding in long videos (93k pairs)

Large dataset for text-based event retrieval in video clips up to 100 minutes long.

  • Video text → text
  • English
  • Chinese
  • Japanese
  • +1
21.2K 1310K–100K rows
DatasetVincent-HKUSTGZ/FragFake

AI Image Editing Detection Dataset via VQA

Image-text pairs for training multimodal models to detect image editing artifacts.

  • Visual question answering
  • English
20.9K 110K–100K rows
DatasetVincentHancoder/RAPTop pick

Scientific Papers and Posters with HTML Markup

25,435 triples of scientific papers, posters, and HTML/CSS templates from AI conferences for poster generation.

  • Image → text
  • Text → image
  • English
20.8K 310K–100K rows
Datasetyutongli2024/EgoExo-Next

EgoExo-Next benchmark for egocentric actions

Visual benchmark for egocentric-exocentric action understanding and future state prediction.

  • Visual question answering
  • Image → text
  • English
20.8K 010K–100K rows
Datasetmlfoundations/MINT-1T-PDF-CC-2023-23Top pick

MINT-1T multimodal corpus chunk (PDF and web data)

Part of a giant open dataset with interleaved texts and images from PDFs and the web.

  • Image → text
  • Text generation
  • English
20.6K 101M–10M rows
DatasetLeoFan01/RoboBench

RoboBench Benchmark for MLLM Robotics Evaluation

English multimodal test suite for checking how well vision-language models reason in robotics scenarios.

  • Robotics
  • Visual question answering
  • English
20.4K 7up to 1K rows
Datasetwendlerc/RenderedTextTop pick

12 million rendered handwriting texts on 3D paper

Synthetic dataset from Stability AI and LAION with English handwritten text images for OCR and generation.

  • Text → image
  • Image → text
  • English
20.3K 5910M–100M rows
Datasetshuolucs/MathSpatialTop pick

Math spatial reasoning benchmark

10,000 math problems with 26K+ geometric diagrams for evaluating MLLM spatial reasoning.

  • Visual question answering
  • Chinese
  • English
20.3K 310K–100K rows
DatasetIPEC-COMMUNITY/EO-Data1.5MTop pick

Multimodal dataset for robotics and embodied AI

Large-scale set of 1 to 10 million examples alternating images, text, and actions for training embodied AI.

  • Robotics
  • Visual question answering
  • Video text → text
  • English
20.3K 241M–10M rows
Datasetwebarena-x/webarena-infinity-trajectories

WebArena web agent trajectories

Dataset of successful browser agent trajectories with step-by-step screenshots and reasoning for agent training.

  • Visual question answering
  • Image → text
20.2K 210K–100K rows
DatasetHumanBehaviorAtlas/human_behavior_atlasTop pick

Multimodal dataset for behavior and emotion analysis

Combined corpus of 16 datasets with video, audio, and text for recognizing emotions, humor, and human state.

  • Video classification
  • Audio classification
  • Text classification
  • English
  • Chinese
20K 4100K–1M rows
Datasettaesiri/SteamScreenshots-Bugs

Steam screenshots with bugs for image-to-text

Array of Steam screenshots ranging from 100 thousand to 1 million examples for image description tasks.

  • Image → text
  • English
20K 2100K–1M rows
DatasetONE-Lab/GUI-World

Video benchmark for multimodal AI agents in interfaces

Dataset featuring videos and questions to evaluate multimodal models in dynamic GUI environments.

  • Question answering
  • Text generation
  • Video text → text
  • English
20K 4610K–100K rows
DatasetSakanaAI/ALE-BenchTop pick

Competitive programming benchmark from Sakana AI and AtCoder

Official set of tasks from AtCoder contests for evaluating AI systems in algorithmic programming with scoring.

  • Image text → text
  • Reinforcement learning
  • Text generation
  • English
  • Japanese
20K 13up to 1K rows
Datasetyangzekang2000/CORAL

Mouse brain neuron reconstruction benchmark

Light microscopy images and annotations for 3D and 2D neuron reconstruction.

  • Image segmentation
  • Image → text
19.9K 11K–10K rows
DatasetAjax102/TaisuTop pick

Large-scale Chinese image-text corpus

A massive collection of Chinese image-text pairs totaling approximately 7.9 TB for image-to-text tasks.

  • Image → text
  • Chinese
19.8K 3100M–1B rows
DatasetAndyson/DefectSpectrum

Industrial defect benchmark with semantic annotation

A collection of manufacturing defect images with segmentation masks and descriptions for quality control systems.

  • Image segmentation
  • Image → text
  • English
19.8K 31K–10K rows
Datasetprinceton-nlp/CharXiv

Benchmark for analyzing complex scientific charts

2,323 scientific charts from arXiv papers to test multimodal model ability to understand complex visualizations.

  • Visual question answering
  • Document question answering
  • Question answering
  • English
19.2K 571K–10K rows
DatasetChartGalaxy/ChartGalaxy

Infographic Charts for VQA

Infographic charts with text annotations for visual understanding and generation tasks.

  • Visual question answering
  • English
18.9K 891K–10K rows
Datasetnvidia/OCR-Synthetic-Multilingual-v1Top pick

NVIDIA synthetic multilingual OCR dataset (with Russian)

Tens of millions of synthetic images for training text recognition in 5 languages, including Russian.

  • Object detection
  • Image → text
  • English
  • Japanese
  • Korean
  • +2
17.5K 6210M–100M rows