Vaani-3 ASR
compact speech recognition with emotion and intent analysis
2 min read
This model from sorika-labs transcribes spoken audio into clean, formatted text while simultaneously detecting the speaker's emotions, tone, sentiment, intent, and urgency in a single pass. Built on a 0.46B-parameter hybrid architecture, it pairs the acoustic encoder (audio processing block) from Qwen/Qwen3-ASR-1.7B with the google/gemma-3-270m text decoder and a dedicated classification module. Unlike standard ASR models, it adds punctuation out of the box, formats numbers, dates, and currencies (ITN — inverse text normalization), and handles mixed-language speech without requiring a language tag upfront. It is built for developers working on voice bots, call centers, and edge devices supporting English and 12 Indian languages.
Key features
- Dual output in a single pass: alongside the transcript, it returns structured JSON scoring tone, emotions, sentiment, call intent, and urgency.
- Ultra-compact and fast: its 0.46B-parameter footprint delivers speeds ranging from 8.1x real-time on a Raspberry Pi 5 up to 62.1x on an NVIDIA A100.
- Support for 12 Indian languages and English: reliably transcribes spontaneous speech and code-switching (switching between Hindi and English mid-sentence).
- Ready-to-use quantized builds: available in GGUF (starting at 283.5 MB for Q4_0) and ONNX INT8 (150 MB) formats for low-end hardware and mobile apps.
What you can do
- Automate call center analytics: transcribe call recordings and immediately route urgent or negative tickets without running a second NLP model.
- Embed into offline assistants: run the Q4_0 or ONNX build on laptops, smartphones, or single-board computers like the Raspberry Pi 5 using roughly 300–460 MB of memory.
- Process long audio recordings: native generation length is extended to 4,096+ tokens (configurable via
max_new_tokens), making it practical for long voice clips.
Example
import torch
from transformers import AutoModel
device = "cuda" if torch.cuda.is_available() else "cpu"
model = AutoModel.from_pretrained("sorika-labs/vaani-3-asr", trust_remote_code=True, torch_dtype=torch.float32).to(device).eval()
transcript = model.transcribe("sample.wav", language="hi", max_new_tokens=4096)
print("Transcript:", transcript)
Read more
DetailsQuality metrics (WER — word error rate, lower is better): AI4Bharat Kathbath (Hindi) — 3.4%, AI4Bharat Lahaja (Hindi) — 8.8%, 12-language average on Google FLEURS — 12.7%.
Latency on 10 seconds of audio: 161 ms on NVIDIA A100 (BF16), 198 ms on RTX 4090 (FP16), 510 ms on Apple M2/M3 (GGUF Q4_0), 1,240 ms on Raspberry Pi 5.
Supported languages: English, Hindi, Bengali, Marathi, Telugu, Tamil, Gujarati, Kannada, Malayalam, Punjabi, Assamese, Odia.
License: Apache 2.0 and Gemma Terms of Use.










Comments