Dataset · ShareGPTVideo
Video frames and instructions for multimodal models
ShareGPTVideo/train_video_and_instruction
Dataset of 900k video frames with English questions and instructions for pretraining and SFT of video-language models.
What's inside
Contains 900k English video frames split into two parts. The first part of 300k frames (from ActivityNet, Vidal, and WebVid) is intended for SFT and DPO, while the remaining 600k frames supplement the pretraining stage dataset. The data is structured for video question-answering (video QA) tasks.
What you can do with it
- SFT and DPO fine-tuning of multimodal models (video-text-to-text) to understand video content
- Pretraining visual-language video stream analysis systems
- Tasks
- Question answering, Video text → text
- Languages
- English
- Downloads this month
- 3.1K
- Likes
- 35
How to load it
from datasets import load_dataset
ds = load_dataset("ShareGPTVideo/train_video_and_instruction")