GeLDA checkpoints for speech emotion recognition
1 min read
The GeLDA collection of checkpoints implements semantically aware generative augmentation of latent data (hidden numerical representations) for zero-shot speech emotion recognition (recognizing emotions in speech without prior training on the target language). Created by researchers Jaesung Bae and Minje Kim, the model improves emotional tone detection in voices using powerful audio models like Whisper and WavLM. The solution relies on latent diffusion models and specialized task adapters. This set is useful for speech technology researchers and voice analysis system developers.
Key features
- Support for popular base audio models, including whisper-large-v3, wavlm-large, and emotion2vec-base.
- Solutions for zero-shot and few-shot emotion recognition scenarios across multiple languages, including Russian.
- Application of latent diffusion in various spaces Z(0), Z(1), and Z(2) for feature generation.
- Ready-to-use full task adapters (Stage-3 task adapters) trained using GeLDA.
What you can do
- Run latent diffusion model inference to generate synthetic emotional features.
- Integrate adapter checkpoints into audio message emotion analysis pipelines.
- Conduct research and experiments on zero-shot speech recognition using Russian-language datasets.
Example
from gelda.diffusion import load_inference_checkpoint
denoiser, config = load_inference_checkpoint("zero_shot/whisper-large-v3/french/fold1/diffusion_z0.safetensors")Read more
DetailsZero-shot setups are supported for various language groups, including Russian with whisper-large-v3.
Folds are structured similarly to the EmoBox framework.
The checkpoints correspond to repetition 0 from the authors' research paper.










Comments