Skip to content

Dataset · HKUSTAudio

Talker-T2AV talking person audio-video dataset

HKUSTAudio/Talker-T2AV-Data

A cleaned multimodal corpus for joint speech and video generation of talking persons in Chinese and English.

What's inside

It contains synchronized audio tracks, video recordings, and motion data packed into tar archives, as well as a CSV index with file paths. The dataset size ranges from 100 thousand to 1 million records in Chinese and English. The dataset was prepared by HKUST researchers to train the Talker-T2AV model.

What you can do with it

  • Training multimodal text-to-audio-video talking avatar generation models
  • Fine-tuning lip-sync and facial expression synchronization systems with synthesized speech
  • Research in joint text-to-speech and text-to-video modeling
Tasks
Text → video, Text → speech
Languages
Chinese, English
Size
100K–1M rows
Downloads this month
1.8K
Likes
7

How to load it

from datasets import load_dataset

ds = load_dataset("HKUSTAudio/Talker-T2AV-Data")