Skip to content

Datasets

Hugging Face datasets with plain-language descriptions: what's inside and what models to train on them. 4,788 in the catalog so far, with new ones added every hour.

Data type

For which models

Find a dataset for your goal

Answered by Gemini 4 · the chat is kept only in this tab

Datasetomegalabsinc/omega-multimodal

Multimodal video dataset from the Bittensor network

A massive video corpus with text descriptions and tabular metadata from the decentralized OMEGA Labs network.

  • Video text → text
  • Video classification
  • Image classification
11.3K 61
Datasetjamescalam/youtube-transcriptions

Technical YouTube Tutorial Transcriptions

Short text snippets from English video tutorials with timestamps for search and RAG.

  • Question answering
  • Text retrieval
  • Visual question answering
  • English
6.5K 44100K–1M rows
Datasetearakely-scale/CuaDriverBench-AgentEnvTop pick

AI agent benchmark for desktop multitasking with humans

Environment and execution logs for evaluating AI agents operating a computer alongside a human user.

  • Reinforcement learning
  • Image text → text
  • English
4.8K 2100K–1M rows
Datasetnvidia/describe-anything-datasetTop pick

Detailed photo and video object descriptions from NVIDIA

English multimodal dataset for training models to describe specific regions in images and videos in detail.

  • Image → text
  • Video text → text
  • English
4.5K 60100K–1M rows
DatasetScaleAI/HSSTop pick

Intuitive Visual Reasoning Benchmark

A collection of images and short videos with questions from Scale AI to evaluate models' ability to understand hidden context.

  • Visual question answering
  • Video text → text
  • English
3.8K 19up to 1K rows
DatasetFBK-MT/MCIFTop pick

Multilingual multimodal benchmark on scientific reports

Human-annotated instruction dataset based on scientific reports: speech, video, and text across four languages to evaluate MLLMs.

  • Automatic speech recognition
  • Question answering
  • Summarization
  • English
  • German
  • Italian
  • +1
3.2K 711K–10K rows
DatasetShareGPTVideo/train_video_and_instruction

Video frames and instructions for multimodal models

Dataset of 900k video frames with English questions and instructions for pretraining and SFT of video-language models.

  • Question answering
  • Video text → text
  • English
3.1K 35
DatasetHarland/OmniVChatTop pick

Native audiovisual dialog dataset

Data from Alibaba and Chinese universities for training and evaluating models processing simultaneous video and voice streams.

  • Video text → text
  • English
  • Chinese
2.6K 581K–10K rows
Datasetnvidia/PhysicalAI-Traffic-Anomaly-ReasoningTop pick

NVIDIA traffic anomaly video and QA dataset

Dataset with videos and text questions from NVIDIA for road anomaly analysis, accepted at NeurIPS 2026.

  • Video text → text
  • Visual question answering
  • English
1.8K 2310K–100K rows
Datasetfacebook/action100m-previewTop pick

Action100M: action video dataset

Large English video dataset with temporal action segments and text descriptions.

  • Video classification
  • Video text → text
  • English
1.2K 152100K–1M rows
DatasetYqjMartin/FLARETop pick

Long video and audio moment retrieval benchmark

Set of 87.7k short video clips with three descriptions each for evaluating multimodal retrieval in long videos.

  • Video text → text
792 2100K–1M rows
Datasetmarkov-ai/computer-use

OSWorld computer agent trajectories

160 successful Gemini agent trajectories on OSWorld tasks with steps and 1.0 scores.

  • Robotics
  • Image → text
728 77up to 1K rows
DatasetHubdata/HubEgo-100KTop pick

First-person video with manual work and IMU data

Over 10,000 hours of first-person real-world workflows with motion sensors for training robots and multimodal models.

  • Robotics
  • Video classification
  • Video text → text
  • English
335 510K–100K rows🔒 on request