TinyVoca DDSP
ultra-lightweight CPU speech synthesis without PyTorch
2 min read
TinyVoca DDSP by TuringLogic is a compact 4.65M-parameter English text-to-speech (TTS) model that outputs 24 kHz mono audio. Built on the TinyVoca architecture, it replaces the standard vocoder with DDSP (Differentiable Digital Signal Processing using 64 harmonics and filtered noise) and was trained from scratch on 29.6 hours of real speaker recordings from the HiFi-TTS dataset. Unlike most modern TTS systems, it requires neither a GPU nor bulky PyTorch: the model weighs around 25 MB including pronunciation dictionaries and runs directly on a CPU or in the browser via ONNX Runtime. It's built for developers creating local voice interfaces, embedded systems, and web apps where a tiny footprint and instant CPU response are critical.
Key features
- Tiny footprint: just 4,648,907 parameters; the FP32 ONNX model takes up 19.52 MB (around 25 MB with pronunciation dictionaries).
- Fast CPU inference: on an Apple M4 Pro, generation via ONNX Runtime 1.32.0 runs at 109.9x real-time speed (RTF 0.0091), synthesizing the first phrase in 0.115 seconds.
- No PyTorch dependency: inference only requires NumPy, ONNX Runtime, SoundFile, and the dizione phonemizer, with browser execution also supported via ONNX Runtime Web.
- Two voices and tempo control: includes two speaker voices (
92and6097), speech rate adjustment, and CPU thread count control.
What you can do
- Embed voice synthesis directly into a web page: run the model client-side via ONNX Runtime Web with zero server GPU costs (web demo source code is included in the repo).
- Add offline speech to a local script or bot: voice short English notifications and replies on a low-end server or laptop with around 500 MB peak RAM usage.
- Train on your own voice: use the open-source PyTorch training pipeline in the repository to train your own compact DDSP model.
Example
hf download TuringLogic/tiny-voca-ddsp --local-dir model \
--include 'onnx/*' 'phonemizer/*' 'onnx_infer.py' 'requirements-onnx.txt'
pip install -r model/requirements-onnx.txt
python model/onnx_infer.py --voice 92 \
--text "Hello, this runs on a CPU." --output hello.wavRead more
DetailsIntelligibility (mean sentence WER via faster-whisper small.en): 13.45% on validation (vs. 4.85% on ground-truth recordings), 7.33% on test, and 1.67% on unseen phrases.
Limitations: English only, single request capped at 512 phoneme tokens and 3,000 acoustic frames (around 32 seconds of audio), full-utterance generation without streaming.
Training: 104,000 steps (37,833 audio clips, ~3h 27m on the training server). Weights are released under the non-commercial CC BY-NC 4.0 license; code is under Apache-2.0.










Comments