Zipformer-65M-vi-en
Bilingual Vietnamese and English speech recognition
2 min read
An offline automatic speech recognition (ASR) model for Vietnamese and English, trained from scratch without relying on existing pretrained checkpoints. Built on the Zipformer2 transducer (RNN-T) architecture using the icefall, k2, and lhotse frameworks, it stays compact at around 80.9M parameters. The model outputs properly capitalized text using a dedicated token in its BPE vocabulary. It is a solid fit for developers who need local recognition of mixed Vietnamese-English speech with inference support via sherpa-onnx.
Key features
- Trained entirely from scratch (random initialization) in four stages: base training followed by three sequential fine-tuning runs.
- Compact footprint of 80,931,511 parameters using a Zipformer2 transducer architecture without auxiliary CTC or attention heads.
- 8,000-token SentencePiece BPE tokenizer includes a dedicated
token for encoding uppercase letters. - Compatible with ONNX export and execution via
sherpa-onnx(the repository includes the checkpoint, tokenizer model, and token table).
What you can do
- Run local offline transcription for Vietnamese and English audio files using
sherpa-onnx. - Use the
model.ptcheckpoint as a base model for further domain-specific fine-tuning in icefall. - Integrate it into voice services or assistants where users switch between Vietnamese and English mid-sentence.
Example
hf download ZAIC26/Zipformer-65M-vi-en model.pt bpe.model tokens.txt \
--local-dir base_models/Zipformer-65M-vi-enRead more
DetailsInput: 80-dimensional fbank features, 16 kHz mono audio.
Operating mode: offline only (non-streaming), without real-time streaming recognition.
The
model.pt checkpoint was created by averaging the last 20 epochs of the final training stage (epochs 6–25).Quality metrics (WER/CER) are not listed on the model card.










Comments