Skip to content

Dataset · Harland

Native audiovisual dialog dataset

Harland/OmniVChat

Data from Alibaba and Chinese universities for training and evaluating models processing simultaneous video and voice streams.

What's inside

The dataset contains 1k to 10k examples with video, audio tracks, and text responses in English and Chinese. In these recordings, the user prompt is embedded directly into the audio stream accompanying the video track. The set was prepared jointly by researchers from Alibaba Group and leading Chinese universities.

What you can do with it

  • Training and SFT of omnimodal models (Omni-LLM) for live video communication
  • Benchmarking systems that synchronously process video and speech without intermediate ASR
Tasks
Video text → text
Languages
English, Chinese
Size
1K–10K rows
Downloads this month
2.6K
Likes
58

How to load it

from datasets import load_dataset

ds = load_dataset("Harland/OmniVChat")