Dataset · Harland
Native audiovisual dialog dataset
Harland/OmniVChat
Data from Alibaba and Chinese universities for training and evaluating models processing simultaneous video and voice streams.
What's inside
The dataset contains 1k to 10k examples with video, audio tracks, and text responses in English and Chinese. In these recordings, the user prompt is embedded directly into the audio stream accompanying the video track. The set was prepared jointly by researchers from Alibaba Group and leading Chinese universities.
What you can do with it
- Training and SFT of omnimodal models (Omni-LLM) for live video communication
- Benchmarking systems that synchronously process video and speech without intermediate ASR
- Tasks
- Video text → text
- Languages
- English, Chinese
- Size
- 1K–10K rows
- Downloads this month
- 2.6K
- Likes
- 58
How to load it
from datasets import load_dataset
ds = load_dataset("Harland/OmniVChat")