Skip to content

Dataset · OpenGVLab

Multimodal video and image instruction annotations

OpenGVLab/VideoChat2-IT

1.9M text annotations in JSON format for training models to answer questions about images and videos.

What's inside

The dataset contains 1.9M instructions and annotations in English in JSON format (media files must be downloaded separately per author instructions). Part of the image data is taken from the M3IT corpus and additionally cleaned of typos and punctuation errors.

What you can do with it

  • Training vision-language models (VLMs) to answer photo and video questions
  • SFT tuning of multimodal assistants for complex instruction execution
Tasks
Visual question answering, Question answering
Languages
English
Size
1M–10M rows
Downloads this month
674
Likes
52
Access
on request on Hugging Face

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("OpenGVLab/VideoChat2-IT")