Pocket TTS Core ML with timestamps: Speech synthesis with precise word-level timing
1 min readupdated
This model is a conversion of Kyutai's Pocket TTS speech engine into the Core ML format for running on Apple devices. The key feature of this version is the added export of word-level timestamps (the exact pronunciation time of each word) directly during speech generation, enabled by the attention weights of specific model heads. This is useful for mobile app and service developers who need synchronization between audio and text, such as for karaoke or real-time subtitle highlighting.
Key features
- Optimized for Core ML to run on Apple devices
- Added pre-softmax attention score output to obtain precise word timestamps
- File structure compatible with the FluidInference ecosystem and FluidAudio library
- Base weights and decoder components fully match the original model from Kyutai
What you can do
- Integrate local speech synthesis into iOS or macOS apps with word-by-word subtitle display
- Create an interactive player with text highlighting during audio playback
- Use the model alongside the FluidAudio library for fast performance on Apple Silicon chips
Example
The model is used via compatible libraries like FluidAudio to generate speech from text while simultaneously obtaining word-level attention timestamps.
Read more
DetailsBase model: kyutai/pocket-tts
Library: coreml
License: CC-BY-4.0
Features: flowlm_step.mlmodelc re-exported with an additional timestamp output










Comments