Tehma/text2sql-qwen2.5-coder-3b-onnx
browser-based SQL generation model
2 min read
This is a fine-tuned version of Qwen2.5-Coder-3B-Instruct (a 3-billion-parameter code language model) trained with QLoRA (quantized low-rank adaptation) on the Spider dataset for text-to-SQL generation. The main highlight of this release is the conversion of weights into a 4-bit ONNX format (q4f16), allowing it to run directly in the browser using Transformers.js and WebGPU hardware acceleration. It is built for developers who want to add local text-to-SQL capabilities without sending user data to a server.
Key features
- Exported to ONNX (a cross-platform model inference format) with 4-bit weight quantization (total file size is around 1.74 GB).
- Runs in the browser via WebGPU at high generation speeds (around 45 tokens per second on an RTX 3060).
- Tailored for generating SQLite queries from a given database schema and a natural language question.
- Local execution eliminates the need to deploy heavy backend servers.
What you can do
- Integrate the model into a Transformers.js web app to build interactive browser-based database interfaces.
- Build a local analytics assistant that translates natural language user queries into SQL code.
Example
import { AutoTokenizer, AutoModelForCausalLM } from "@huggingface/transformers";
const id = "Tehma/text2sql-qwen2.5-coder-3b-onnx";
const tokenizer = await AutoTokenizer.from_pretrained(id);
const model = await AutoModelForCausalLM.from_pretrained(id, { dtype: "q4f16", device: "webgpu" });
const inputs = tokenizer.apply_chat_template(messages, { add_generation_prompt: true, return_dict: true });
const out = await model.generate({ ...inputs, max_new_tokens: 256, do_sample: false });Read more
DetailsBenchmarks: On the Spider dataset (first 200 questions of the dev split), the q4f16 ONNX model achieves 64.0% execution accuracy.
Hardware requirements: A WebGPU-compatible device (for example, an RTX 3060 GPU delivers ~45 tokens/s).
Limitations: Trained on the Spider dataset (English only, SQLite dialect, academic schemas); not tested on other SQL dialects or real-world business databases.
License: Qwen Research License (non-commercial use only).










Comments