Skip to content

Datasets

Hugging Face datasets with plain-language descriptions: what's inside and what models to train on them. 4,788 in the catalog so far, with new ones added every hour.

Data type

For which models

Find a dataset for your goal

Answered by Gemini 4 · the chat is kept only in this tab

DatasetOriginFlow-AI/origindata-preview-sampleTop pick

Hand-Object Interaction: Video, Poses, and Force

A robotics multimodal dataset featuring hand motion recordings and applied force in real-world tasks.

  • Robotics
6.4K 2510M–100M rows🔒 on request
DatasetSKY-Piano/SKY-Piano

Multimodal Piano Performance Dataset

Synchronized audio, MIDI, motion capture, and fingering data for piano performance analysis.

    6.1K 10🔒 on request
    DatasetHFUT-MMH-Lab/multimodal_mental_health_dataset_MMH

    Multimodal dataset for mental disorder diagnosis

    Participant audio, video, and texts in English and Chinese with clinically confirmed diagnoses for distinguishing disorders.

    • Audio classification
    • Video classification
    • Text classification
    • English
    • Chinese
    5.3K 510K–100K rows
    Datasetearakely-scale/CuaDriverBench-AgentEnvTop pick

    AI agent benchmark for desktop multitasking with humans

    Environment and execution logs for evaluating AI agents operating a computer alongside a human user.

    • Reinforcement learning
    • Image text → text
    • English
    4.8K 2100K–1M rows
    Datasetnvidia/describe-anything-datasetTop pick

    Detailed photo and video object descriptions from NVIDIA

    English multimodal dataset for training models to describe specific regions in images and videos in detail.

    • Image → text
    • Video text → text
    • English
    4.5K 60100K–1M rows
    DatasetControlNet/AV-Deepfake1M

    Large-scale English audiovisual deepfake dataset

    Millions of English videos for training and evaluating deepfake detection systems.

    • Video classification
    • English
    4.2K 291M–10M rows🔒 on request
    Datasetunitreerobotics/G1_Dex1_Arrange_Surgical_KitsTop pick

    Unitree G1 robot multimodal data for kit assembly

    Camera and sensor recordings from the Unitree G1 humanoid robot performing surgical kit arrangement tasks.

    • Robotics
    4K 3100K–1M rows
    Datasetnvidia/PhysicalAI-WorldModel-Synthetic-Embodied-Robot-ScenesTop pick

    NVIDIA synthetic robot scene videos

    Large corpus of synthetic videos with robots from NVIDIA simulators for training physical world models.

      3.9K 41100K–1M rows
      DatasetNU-World-Model-Embodied-AI/phyground

      Physics benchmark in generated video

      250 prompts with initial frames and labels for 13 laws of physics to test the physical validity of video generation models.

      • Text → video
      • Image → video
      • English
      3.9K 48up to 1K rows
      DatasetScaleAI/HSSTop pick

      Intuitive Visual Reasoning Benchmark

      A collection of images and short videos with questions from Scale AI to evaluate models' ability to understand hidden context.

      • Visual question answering
      • Video text → text
      • English
      3.8K 19up to 1K rows
      DatasetMIMUW-Robotics/kine2go

      Kine2Go kinematic dataset for the Go2 robot

      A kinematic dataset of Unitree Go2 quadruped robot movements for policy training.

      • Robotics
      • Reinforcement learning
      3.7K 11up to 1K rows
      DatasetMCG-NJU/SportsAction

      Spatio-temporal sports action annotation dataset

      MultiSports video dataset featuring multi-person action annotations in space and time.

      • Image classification
      • Object detection
      • Other
      • English
      3.6K 511K–10K rows🔒 on request
      Datasetgenrobot2025/Gen-EgoData

      Egocentric video and SLAM data for robotics

      Demo sample of 500 first-person videos (4.23 hours) with motion trajectories for training home robots.

      • Robotics
      • Reinforcement learning
      • English
      • Chinese
      3.5K 38100B–1T rows
      DatasetAlayaLab/WildWorld

      Video dataset for ARPG genre game world models

      Large video dataset with actions and states for training generative game world models.

      • Text → video
      • Image → video
      • Video → video
      • English
      3.4K 15🔒 on request
      DatasetSynthForensics/SynthForensicsTop pick

      Deepfake detection benchmark for generated video

      Synthetic human video dataset for evaluating and training detectors for modern video generators.

      • Video classification
      • English
      3.3K 1510K–100K rows🔒 on request
      DatasetYootta/World-SimReady-Home

      SimReady home scene assets for robots

      CAD assets and scenes with configured physics for home robot simulation.

      • Robotics
      • English
      3.2K 112🔒 on request
      DatasetFBK-MT/MCIFTop pick

      Multilingual multimodal benchmark on scientific reports

      Human-annotated instruction dataset based on scientific reports: speech, video, and text across four languages to evaluate MLLMs.

      • Automatic speech recognition
      • Question answering
      • Summarization
      • English
      • German
      • Italian
      • +1
      3.2K 711K–10K rows
      DatasetShareGPTVideo/train_video_and_instruction

      Video frames and instructions for multimodal models

      Dataset of 900k video frames with English questions and instructions for pretraining and SFT of video-language models.

      • Question answering
      • Video text → text
      • English
      3.1K 35