sherpa-onnx-whisper-small - Whisper model export for ONNX and Unreal Engine
2 min readupdated
This model is a version of the well-known Whisper Small architecture, repackaged into the ONNX format (an open format for representing machine learning models) for use in the Sentients plugin audio core for Unreal Engine. It is designed for speech-to-text translation across 99 languages and optimized for fast performance across various runtimes, including CPU, DirectML, and TensorRT. The package includes both original floating-point (fp32) files and optimized half-precision (fp16) encoder versions. This tool is useful for Unreal Engine game and application developers who need reliable speech recognition directly inside the engine.
Key features
- Export of the popular OpenAI Whisper Small architecture with support for 99 languages
- Support for various runtimes: CPU, DirectML, TensorRT, and HLSL
- Availability of half-precision (fp16) files to speed up encoder performance
- Integration with voice input components and the Silero VAD plugin for silence filtering
What you can do
- Implement voice control or speech recognition in Unreal Engine projects via the Sentients plugin
- Use optimized ONNX files for inference on various hardware platforms without PyTorch dependencies
- Set up local audio transcription using fixed shapes for faster execution in specialized runtimes
Example
The model is used through Sentients plugin components in Unreal Engine, selected by the identifier EAudioModel::WhisperSmall, accepting CoreAudioInput as input and returning CoreTranscriptOutput along with the VAD module for voice activity detection.
Read more
DetailsFormat: ONNX
Precision: fp32 and fp16
Base model: openai/whisper-small (exported via csukuangfj/sherpa-onnx-whisper-small)
Features: fixed shapes for runtimes, SHA-256 file pinning in the Sentients directory










Comments