Tehma/text2sql-qwen2.5-coder-1.5b-onnx - an AI model for browser-based SQL generation
2 min readupdated
This model is an optimized version of Qwen2.5-Coder-1.5B-Instruct, fine-tuned on the Spider dataset for translating natural language queries into SQL code. The model has been converted to the ONNX format (a format for running models across different platforms) with q4f16 quantization (4-bit weights to reduce size) specifically to run directly in the browser using the Transformers.js library powered by WebGPU. It is ideal for developers who want to add local SQL generation to their web applications without sending data to a server.
Key features
- Based on the efficient Qwen2.5-Coder-1.5B-Instruct language model.
- Exported to the ONNX format to run in the browser via Transformers.js and WebGPU.
- Uses 4-bit weight quantization (q4f16) with a size of 877 MB to save memory.
- Fine-tuned using QLoRA on the Spider dataset for text-to-SQL tasks.
What you can do
- Embed text-to-SQL generation directly into the client side of a web application.
- Build a local developer assistant that runs offline in the browser.
- Test query execution on SQLite without setting up a backend.
Example
import { AutoTokenizer, AutoModelForCausalLM } from "@huggingface/transformers";
const id = "Tehma/text2sql-qwen2.5-coder-1.5b-onnx";
const tokenizer = await AutoTokenizer.from_pretrained(id);
const model = await AutoModelForCausalLM.from_pretrained(id, { dtype: "q4f16", device: "webgpu" });
const inputs = tokenizer.apply_chat_template(messages, { add_generation_prompt: true, return_dict: true });
const out = await model.generate({ ...inputs, max_new_tokens: 256, do_sample: false });
Read more
DetailsBenchmarks: 71.4% execution accuracy on the full Spider dataset for the base version.
Requirements: Model size is 877 MB, delivers about 62 tokens per second in the browser (RTX 3060).
Limitations: Trained only on English questions and the SQLite dialect, not tested on real business schemas.
License: Apache 2.0.










Comments