byt5-fuzzy-phoneme-decode-tiny
a compact decoder for restoring corrupted phoneme sequences
2 min read
Built on Google's byte-level ByT5 architecture, this tiny 17M-parameter model tackles an unusual task: converting distorted or corrupted phonemes ("surface phonology") back into their canonical form ("canonical phonology"). It was trained on 1.67 million synthetic pairs generated by a phonological process calibrator using PhonBank data across six languages (English, German, French, Italian, Dutch, and Spanish). The model is useful for speech researchers, AAC (augmentative and alternative communication) developers, and anyone working with challenging speech recognition or speech disorders.
Key features
- Lightweight: Weighs in at around 70 MB with just 17M parameters.
- Multilingual: Supports English, German, French, Italian, Dutch, and Spanish.
- ONNX variants: Available in fp32 and int8 quantized (dynamic-QInt8) versions, shrinking the file size by roughly 4x with no loss in quality.
- Targeted training: Trained to invert real-world speech processes, including syllable deletion, assimilation, devoicing, and more.
What you can do
- Integrate into AAC: Use it as an input component for alternative communication systems to handle slurred or distorted speech.
- Fix transcriptions: Post-process noisy phoneme sequences output by speech recognition models.
- Experiment with ONNX: Drop the optimized fp32 or int8 builds directly into your audio and text processing pipelines.
Example
Example of the model fixing devoicing in German:: ʔ a m p uː t iː ɡ t ə n → ʔ a m p uː t iː ʁ t ə n
Read more
Details• Exact recovery accuracy: ~49% on the gold test set (n=4000) across all corruption levels.
• Metrics: Phone TER ranges from 0.06 to 0.12 depending on the language.
• Hardware: Ultra-lightweight (~70 MB in fp32, even smaller in ONNX int8) with minimal VRAM requirements.
• Data: Trained on synthetic pairs from voicegarden-lexicons calibrated against PhonBank.










Comments