Skip to content

Datasets

Hugging Face datasets with plain-language descriptions: what's inside and what models to train on them. 4,788 in the catalog so far, with new ones added every hour.

Data type

For which models

Find a dataset for your goal

Answered by Gemini 4 · the chat is kept only in this tab

DatasetLeoFan01/RoboBench

RoboBench Benchmark for MLLM Robotics Evaluation

English multimodal test suite for checking how well vision-language models reason in robotics scenarios.

  • Robotics
  • Visual question answering
  • English
20.4K 7up to 1K rows
Datasetwendlerc/RenderedTextTop pick

12 million rendered handwriting texts on 3D paper

Synthetic dataset from Stability AI and LAION with English handwritten text images for OCR and generation.

  • Text → image
  • Image → text
  • English
20.3K 5910M–100M rows
DatasetIPEC-COMMUNITY/EO-Data1.5MTop pick

Multimodal dataset for robotics and embodied AI

Large-scale set of 1 to 10 million examples alternating images, text, and actions for training embodied AI.

  • Robotics
  • Visual question answering
  • Video text → text
  • English
20.3K 241M–10M rows
DatasetHumanBehaviorAtlas/human_behavior_atlasTop pick

Multimodal dataset for behavior and emotion analysis

Combined corpus of 16 datasets with video, audio, and text for recognizing emotions, humor, and human state.

  • Video classification
  • Audio classification
  • Text classification
  • English
  • Chinese
20K 4100K–1M rows
DatasetSakanaAI/ALE-BenchTop pick

Competitive programming benchmark from Sakana AI and AtCoder

Official set of tasks from AtCoder contests for evaluating AI systems in algorithmic programming with scoring.

  • Image text → text
  • Reinforcement learning
  • Text generation
  • English
  • Japanese
20K 13up to 1K rows
Datasetyangzekang2000/CORAL

Mouse brain neuron reconstruction benchmark

Light microscopy images and annotations for 3D and 2D neuron reconstruction.

  • Image segmentation
  • Image → text
19.9K 11K–10K rows
DatasetAjax102/TaisuTop pick

Large-scale Chinese image-text corpus

A massive collection of Chinese image-text pairs totaling approximately 7.9 TB for image-to-text tasks.

  • Image → text
  • Chinese
19.8K 3100M–1B rows
Datasetprinceton-nlp/CharXiv

Benchmark for analyzing complex scientific charts

2,323 scientific charts from arXiv papers to test multimodal model ability to understand complex visualizations.

  • Visual question answering
  • Document question answering
  • Question answering
  • English
19.2K 571K–10K rows
DatasetChartGalaxy/ChartGalaxy

Infographic Charts for VQA

Infographic charts with text annotations for visual understanding and generation tasks.

  • Visual question answering
  • English
18.9K 891K–10K rows
Datasetgoogle/RSRCCTop pick

Google Remote Sensing Change Captioning Benchmark

A Google Research dataset with multi-temporal aerospace image pairs and questions about terrain changes.

  • Visual question answering
  • Image text → text
  • Multiple choice
  • English
16.2K 41100K–1M rows
DatasetMathLLMs/MathVision

MATH-Vision benchmark for multimodal mathematics

Visual math problems with images and text for evaluating VLMs.

  • Question answering
  • Multiple choice
  • Visual question answering
  • English
15.7K 1841K–10K rows
DatasetMAmmoTH-VL/MAmmoTH-VL-Instruct-12M

12M visual instructions for multimodal models

Image-instruction pairs for training visual assistants in English.

  • Visual question answering
  • Question answering
  • English
15.4K 6710M–100M rows
DatasetOpen-Bee/Honey-Data-15M

Bee multimodal image-text corpus

15M English image-text pairs for training and evaluating open multimodal models.

  • Image text → text
  • English
15K 12110M–100M rows
DatasetQwen/Qwen-Image-BenchTop pick

Qwen Image Generation Benchmark

An English and Chinese image-text dataset for deep evaluation of Text-to-Image models.

  • Image → text
  • English
  • Chinese
15K 531K–10K rows
DatasetShareGPT4Video/ShareGPT4Video

ShareGPT4Video: video captions

4.8M multimodal GPT-4-Vision video descriptions for LVLM and text-to-video models.

  • Visual question answering
  • Question answering
  • English
14K 21610K–100K rows
DatasetShadenA/MathNet

Multilingual math problems with images

Problems from countries and competitions with images and text for QA and solution generation.

  • Question answering
  • Text generation
  • Image → text
  • English
  • Portuguese
  • Spanish
  • +9
13.3K 10510K–100K rows
DatasetMME-Benchmarks/Video-MME-v2

Video-MME v2 Video Understanding Benchmark

Updated benchmark with English videos and annotations for comprehensive evaluation of video models.

  • Video text → text
  • English
13.3K 521K–10K rows
Datasetwikimedia/wit_base

Multilingual image-text pairs from Wikipedia

Image and text connections from Wikipedia across many languages for training and evaluating multimodal retrieval.

  • Image → text
  • Text retrieval
  • Afrikaans
  • Aragonese
  • Arabic
  • +9
12.5K 77100K–1M rows
Datasetnyu-visionx/CV-Bench

CV-Bench visual benchmark for multimodal LLMs

1K to 10K test examples with images and questions to evaluate vision capabilities in multimodal models.

  • Visual question answering
  • English
12.2K 491K–10K rows