omniASR-CTC-3B
a large multilingual speech recognition model
2 min readupdated
This model transcribes spoken audio into text across multiple languages. It is based on the omniASR_CTC_3B checkpoint from Facebook Research's OmniLingual project, converted from the fairseq2 framework into the transformers library format using the Wav2Vec2ForCTC architecture (a speech recognition model that handles variable-length audio). Unlike the original release, this port lets you skip setting up fairseq2 and run inference with standard Hugging Face tools. It is a solid fit for developers building voice services, transcription tools, and multilingual audio processing pipelines.
Key features
- Packs 3.08 billion parameters across 60 encoder layers, with a hidden size of 2048 and 32 attention heads.
- Supports multilingual transcription and outputs CTC logits over a 9,812-token SentencePiece vocabulary.
- Native
transformerscompatibility: model loading and decoding work out of the box via standardAutoProcessorandWav2Vec2ForCTCclasses. - The author notes that full output parity verification against the original fairseq2 checkpoint has not been performed.
What you can do
- Integrate it into transcription pipelines for multilingual podcasts, interviews, and voice notes.
- Use it as a baseline ASR backend in
transformers-based projects without tying your stack to the fairseq2 ecosystem. - Benchmark its recognition accuracy on your target languages and compare the results against the original OmniLingual build.
Example
from transformers import Wav2Vec2ForCTC, AutoProcessor
import torch, torchaudio
processor = AutoProcessor.from_pretrained("nolimitsxl/omniASR-CTC-3B")
model = Wav2Vec2ForCTC.from_pretrained("nolimitsxl/omniASR-CTC-3B").eval()
waveform, sr = torchaudio.load("audio.wav")
inputs = processor(waveform.squeeze().numpy(), sampling_rate=16_000, return_tensors="pt")
print(processor.decode(torch.argmax(model(**inputs).logits, dim=-1)[0]))
Read more
DetailsArchitecture: Wav2Vec2ForCTC, 60 layers, FFN intermediate 8192.
Input audio sample rate: 16,000 Hz.
Source: port of the omniASR_CTC_3B checkpoint from facebookresearch/omnilingual-asr.
Limitation: parity verification was not run during conversion.










Comments