SQLBuddy Qwen2.5-Coder-1.5B LoRA - text-to-SQL generation adapter
2 min read
This model is a LoRA (small add-on for a base model) built on Qwen/Qwen2.5-Coder-1.5B-Instruct, fine-tuned to convert text questions into SQL queries. It takes a system message, table schema (CREATE TABLE), and the question itself through the Qwen chat template, and outputs a single SQLite query. Unlike the general-purpose base model, this adapter is narrowly specialized for the query format from the Spider dataset. The solution is useful for developers, analysts, and students looking to prototype text-to-SQL systems on local hardware.
Key features
- Text-to-SQL specialization: trained on 6,560 examples from the Spider benchmark with loss calculated only on the final SQL code.
- Architecture configuration: an adapter with rank r=16 and alpha=32 targeting attention and MLP projections (
q_proj,k_proj,v_proj,o_proj,gate_proj,up_proj,down_proj). - Compact size: out of 1.56 billion parameters, only 18.46 million were trained, making download and inference easy.
- Simple integration: easily combined with the base model using the
peftlibrary.
What you can do
- Build a local assistant to generate SQLite queries based on your database schema.
- Integrate the model into a chatbot or internal service to quickly translate user questions into SQL.
- Use the adapter in educational projects to test and evaluate text-to-SQL models.
Example
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
tok = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-Coder-1.5B-Instruct")
model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-Coder-1.5B-Instruct")
model = PeftModel.from_pretrained(model, "ibrahimsalman022/sqlbuddy-qwen2.5-coder-1.5b-lora").merge_and_unload()Read more
DetailsBase model: Qwen/Qwen2.5-Coder-1.5B-Instruct (1.56B parameters)
Training: 2 epochs (820 steps), lr 2e-4 cosine, batch 16, max length 1536, fp16
Dataset: Spider (6,560 train examples, 352 val examples across 8 held-out databases)
Loss metrics: final train loss 0.0896, final eval loss 0.2227
Training hardware: Tesla T4, peak VRAM usage 5.43 GB










Comments