Skip to content

Dataset · ShareGPTVideo

Video frames and instructions for multimodal models

ShareGPTVideo/train_video_and_instruction

Dataset of 900k video frames with English questions and instructions for pretraining and SFT of video-language models.

What's inside

Contains 900k English video frames split into two parts. The first part of 300k frames (from ActivityNet, Vidal, and WebVid) is intended for SFT and DPO, while the remaining 600k frames supplement the pretraining stage dataset. The data is structured for video question-answering (video QA) tasks.

What you can do with it

  • SFT and DPO fine-tuning of multimodal models (video-text-to-text) to understand video content
  • Pretraining visual-language video stream analysis systems
Tasks
Question answering, Video text → text
Languages
English
Downloads this month
3.1K
Likes
35

How to load it

from datasets import load_dataset

ds = load_dataset("ShareGPTVideo/train_video_and_instruction")