Whisper large-v3 Core ML (8-bit)
local speech-to-text on Apple devices at half the size
2 min readupdated
This is a Core ML conversion of the popular openai/whisper-large-v3 speech recognition model built for the WhisperKit framework. Its weights are quantized to 8-bit (compressed for faster execution and lower memory use), letting it run on the Apple Neural Engine at roughly 1.56 GB instead of the standard float16 version's 3.1 GB. It handles local transcription (converting audio to text) without sending recordings to the cloud while preserving the original model's accuracy. It is aimed at macOS, iOS, and iPadOS developers who need fast, high-quality on-device speech recognition.
Key features
- Compact footprint: takes up roughly 1.56 GB in total (648 MB for the audio encoder and 910 MB for the text decoder), half the size of the float16 version.
- Fast performance on Mac: averages a 0.21 RTF (processing time per second of audio), compared to 0.55 for the float16 build.
- Virtually no accuracy loss: averages an 8.0% WER across 10 test sets, versus 7.9% for the original float16 version.
- Production-proven: already powers the Sagt app on Mac, iPhone, and iPad.
What you can do
- Build local voice recorders or voice memo transcription into iPhone, iPad, and Mac apps using the WhisperKit library.
- Add offline transcription for calls, interviews, and podcasts in English, German, Dutch, Spanish, and French with zero cloud API costs.
Example
let folder = try await WhisperKit.download(
variant: "openai_whisper-large-v3_8bit",
from: "Selbyggen/whisper-large-v3-coreml"
)
let kit = try await WhisperKit(WhisperKitConfig(modelFolder: folder.path))
let result = try await kit.transcribe(audioPath: "recording.m4a")
Read more
DetailsAverage WER across 10 datasets (FLEURS and VoxPopuli for en, de, nl, es, fr): 8.0% (vs. 7.9% for float16 and 8.7% for large-v3 turbo).
Speed: average RTF on Mac is 0.21 (lower is faster; float16 is 0.55).
Quantization: per-channel linear symmetric int8 via coremltools (8-bit weights, 16-bit activations, float16 mel spectrogram).
Compatibility: Apple devices supporting Core ML / Apple Neural Engine via WhisperKit; Apache 2.0 license.










Comments