Qwen3-ASR-1.7B (FP8 W8A8)
fast quantized model for speech recognition
2 min readupdated
This model is an optimized quantized version (FP8 W8A8) of the modern Qwen3-ASR-1.7B speech recognition and acoustic system. It is designed for high-quality transcription of audio files, songs, and conversational speech, along with on-the-fly language identification for 30 languages and 22 dialects (including Russian). Thanks to targeted quantization of the decoder linear layers, the model size is reduced by more than a third while maintaining high accuracy. The model supports running via convenient transformers and vLLM (high-speed generation server) backends for efficient inference.
Key features
- FP8 W8A8 quantization: decoder linear layer weights are compressed to the FP8 format, while the audio encoder and projector remain in BF16 to preserve acoustic accuracy.
- Support for 52 languages and dialects: confident recognition of speech, accents, and songs in Russian, English, Chinese, and other languages.
- Unified inference pipeline: support for batch processing, streaming mode, and standalone transcription of long audio recordings.
- vLLM compatibility: high processing speed and the ability to deploy via a standard OpenAI-compatible server.
What you can do
- Run a local transcription service: deploy a speech recognition server using vLLM to process audio streams in real time.
- Transcribe long podcasts and videos: automate the transcription of multi-hour audio recordings with high speed and minimal VRAM requirements.
- Integrate into voice bots: use the model as a backend for recognizing user speech in multimodal applications.
Example
import torch
from qwen_asr import Qwen3ASRModel
model = Qwen3ASRModel.from_pretrained(
"lly0571/Qwen3-ASR-1.7B-FP8-W8A8",
dtype=torch.bfloat16,
device_map="cuda:0"
)
results = model.transcribe(
audio="[https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-ASR-Repo/asr_en.wav](https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3-ASR-Repo/asr_en.wav)",
language=None
)
print(results[0].text)Read more
DetailsWeight size: 2.48 GB (-34.6% compared to the baseline BF16 version).
Recognition quality (WER on LibriSpeech test-clean): 1.64% (virtually no loss against 1.62% for the original).
Optimization: 196 decoder linear layers quantized using LLM Compressor tools.










Comments