Nemotron 3.5 ASR Streaming 0.6B (CoreML)
fast speech recognition for Apple Silicon
2 min readupdated
A ready-to-use build of NVIDIA's streaming automatic speech recognition (ASR) model, converted to CoreML to run locally on Apple Silicon via the Apple Neural Engine (ANE — Apple's built-in neural processor). Based on a 0.6B-parameter Conformer + RNN-T architecture, this version skips heavy server setups—it uses post-training quantization and comes pre-compiled to .mlmodelc so you can drop it straight into iOS 17+ and macOS 14+ apps. Built for iOS and macOS developers who need real-time offline transcription.
Key features
- Two vocabulary options: a lightweight
latinbundle (2,828 tokens covering en, es, fr, it, pt, de) and a fullmultilingualbundle (13,087 tokens supporting Chinese, Japanese, and over 100 other languages viaprompt_id). - Four latency modes: pre-packaged bundles for 0.56s, 1s, 2s, and 4s chunks let you trade off responsiveness against throughput (2s is recommended by default).
- Lossless mixed-precision quantization: outer encoder layers are compressed to INT8 and middle layers to 6-bit, cutting encoder size by roughly 55% compared to FP16 without hurting accuracy.
- M2 and M3 fixes: the repo includes alternate encoder variants (
encoder-v2and a splitencoder-split) that compile cleanly on M2 and M3 Neural Engines without falling back to the CPU.
What you can do
- Offline captions and voice recorders on iPhone and Mac: embed the compiled bundles into mobile or desktop apps via the FluidAudio library for instant transcription without an internet connection.
- Voice assistants and local agents: use the 1s tier (for European languages) or 2s tier (for multilingual setups) as a fast speech-input front end on a MacBook.
- Batch transcription of long recordings: plug in the 4s bundle for high-speed offline transcription of lectures and interviews.
Example
fluidaudiocli nemotron-multilingual-transcribe \
--input audio.wav \
--model-dir latin/2240ms \
--language de-DE
Read more
Details• Speed on Apple M5 Pro (2s chunk): 130–150 RTFx for Latin languages (4.80%–9.83% WER on FLEURS) and 80–89 RTFx for the multilingual version; on LibriSpeech test-clean for English, it hits 120 RTFx at 3.67% WER.
• The full-vocabulary version (
multilingual) is sensitive to chunk size: on 0.56s and 1s chunks, speed for CJK drops to 21–27 RTFx, so stick to the 2s mode for those languages.• Compatibility: iOS 17+ / macOS 14+, distributed under the commercial OpenMDW-1.1 license (repo access is granted upon request in the author's Discord).










Comments