LoRA for Qwen3.5-4B: SQL generation in Russian, English, and Uzbek
2 min readupdated
This LoRA adapter (a lightweight add-on for a base model) is fine-tuned to turn natural language questions into ready-to-run PostgreSQL queries. Built on top of Qwen/Qwen3.5-4B, the model supports prompts in English, Russian, and Uzbek (both Latin and Cyrillic scripts). It beats the base model on SQL execution accuracy and skips internal thinking traces, outputting the final query right away. This release is useful for developers and data analysts working with multilingual databases.
Key features
- Improves accuracy on the uzbird benchmark to 58.6%, up from 50.2% for the base model.
- Supports English, Russian, and Uzbek (Latin and Cyrillic scripts).
- Trained exclusively on target SQL queries without intermediate reasoning steps.
- Configured with LoRA rank 32 across all linear layers of the model.
What you can do
- Plug the adapter into Text-to-SQL pipelines to automatically generate database queries.
- Integrate the model into analytics chatbots to handle queries in Russian or Uzbek.
- Run inference using the peft library or merge the weights to serve via vLLM.
Example
import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("Qwen/Qwen3.5-4B")
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.5-4B", dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(base, "mirzakhojayev/t2sql-uz-qwen3.5-4B")
system = ("You are an expert data analyst. Write one SQL query that answers the question for the given database. "
"Use only tables and columns from the schema, and the SQL dialect named in the prompt. "
"Briefly explain your reasoning, then give the final query in a ```sql code block.")
user = """Dialect: PostgreSQL
Schema:
CREATE TABLE schools (
cdscode TEXT PRIMARY KEY,
county TEXT,
charter BIGINT
);
Question: Los Angeles okrugida nechta charter maktab bor?"""
text = tok.apply_chat_template([{"role": "system", "content": system}, {"role": "user", "content": user}],
tokenize=False, add_generation_prompt=True, enable_thinking=False)
inputs = tok(text, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=512, do_sample=True, temperature=0.7, top_p=0.8, top_k=20)
print(tok.decode(out[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))
Read more
DetailsMethod: LoRA, rank 32, alpha 64, dropout 0.05 across all linear layers
Dataset: 26,249 examples from uzbird train (BIRD train and SynSQL-2.5M)
Hardware: 1× A100 40GB, training took 4.7 hours (peak VRAM 32.7 GB)
License: Apache 2.0










Comments