Ternary-Bonsai-2-27B-PQ2_0-ONNX-WebGPU
ONNX export of a large language model for WebGPU
2 min readupdated
This release exports the weights of the Ternary-Bonsai-2-27B language model to the ONNX format with WebGPU execution provider support. Built on the Qwen3.8-27B architecture, the model uses 2-bit PQ2 quantization and FP16 for floating-point tensors without losing weight structure. The package includes all 64 decoder layers, tokenizer files, and configs needed to run text generation in supported environments. It is aimed at developers experimenting with running large models locally or in web environments.
Key features
- Exported to ONNX Runtime with WebGPU execution provider support
- 2-bit PQ2 matrix quantization while keeping FP16 for the recurrent state
- Complete set of 64 decoder layers and tokenizer extracted from a GGUF checkpoint
- Includes generation configuration and the chat_template.jinja chat template
What you can do
- Integrate the model into web apps and services powered by ONNX Runtime WebGPU
- Run local text generation on WebGPU-compatible hardware
- Use the model files and weight shards in custom inference pipelines
Example
To use this export, place all files (including the four weight shards totaling around 7.25 GB, model.onnx, and genai_config.json) into a single directory and run them via ONNX Runtime with the WebGPU provider.
Read more
DetailsBase model: Qwen3.8-27B
Architecture: 64 decoder layers, hidden size 5,120
Quantization: MatMulNBits, bits=2, block_size=128
Weight size: 4 shard files totaling around 7.25 GB










Comments