Skip to content

Datasets

Hugging Face datasets with plain-language descriptions: what's inside and what models to train on them. 4,788 in the catalog so far, with new ones added every hour.

Data type

For which models

Find a dataset for your goal

Answered by Gemini 4 · the chat is kept only in this tab

DatasetPawChrona/PawChrona-MMQA

Pet Health and Behavior Multimodal Q&A Dataset

49K English and Chinese questions about cats and dogs with images, audio, video, and text.

  • Visual question answering
  • Question answering
  • English
  • Chinese
27.4K 11100K–1M rows🔒 on request
DatasetMLVU/MVLU

Long video understanding benchmark (MLVU)

English-language test set and annotations for evaluating neural networks on long video analysis.

  • Question answering
  • English
26.9K 48🔒 on request
DatasetMC7ever/hf-training-corpus

Multimodal HF Corpus of 92K Datasets

Large-scale extraction of texts, images, audio, and video from open datasets in a unified parquet format.

  • Text generation
  • Image → text
  • Automatic speech recognition
  • English
26.8K 1100K–1M rows
DatasetHumanBehaviorAtlas/human_behavior_atlasTop pick

Multimodal dataset for behavior and emotion analysis

Combined corpus of 16 datasets with video, audio, and text for recognizing emotions, humor, and human state.

  • Video classification
  • Audio classification
  • Text classification
  • English
  • Chinese
20K 4100K–1M rows
DatasetONE-Lab/GUI-World

Video benchmark for multimodal AI agents in interfaces

Dataset featuring videos and questions to evaluate multimodal models in dynamic GUI environments.

  • Question answering
  • Text generation
  • Video text → text
  • English
20K 4610K–100K rows
DatasetShareGPT4Video/ShareGPT4Video

ShareGPT4Video: video captions

4.8M multimodal GPT-4-Vision video descriptions for LVLM and text-to-video models.

  • Visual question answering
  • Question answering
  • English
14K 21610K–100K rows
Datasetjamescalam/youtube-transcriptions

Technical YouTube Tutorial Transcriptions

Short text snippets from English video tutorials with timestamps for search and RAG.

  • Question answering
  • Text retrieval
  • Visual question answering
  • English
6.5K 44100K–1M rows
DatasetFBK-MT/MCIFTop pick

Multilingual multimodal benchmark on scientific reports

Human-annotated instruction dataset based on scientific reports: speech, video, and text across four languages to evaluate MLLMs.

  • Automatic speech recognition
  • Question answering
  • Summarization
  • English
  • German
  • Italian
  • +1
3.2K 711K–10K rows
DatasetShareGPTVideo/train_video_and_instruction

Video frames and instructions for multimodal models

Dataset of 900k video frames with English questions and instructions for pretraining and SFT of video-language models.

  • Question answering
  • Video text → text
  • English
3.1K 35