Qwen2.5-Coder-3B for RAG engineering: a three-tier fine-tune
1 min readupdated
The gaurav0506/rag-sft-qwen2.5-coder-3b model is the result of supervised fine-tuning (SFT) of the base Qwen2.5-Coder-3B model for Retrieval-Augmented Generation tasks. The authors explored code evolution from the base version through domain-specific continued pre-training (CPT) to full SFT on 3,000 repository examples. The new model completely eliminates infinite generation loops and unhashable array errors, ensuring 100% AST syntactic validity. This tool will be useful for developers of local AI agents and documentation search systems.
Key features
- 100% Python AST syntactic validity across 10 autonomous search tasks.
- Complete elimination of infinite recursive code generation loops.
- Strict adherence to ChatML chat format boundaries and modern SDK standards.
- Token accuracy boost up to 74.85% compared to the base version.
What you can do
- Build local hybrid search pipelines with BM25 and RRF ranking.
- Write queries for Qdrant and vector databases without namespace hallucinations.
- Implement Python RAG evaluation metrics such as nDCG and MRR.
Example
Use case: ask the model to write a hybrid search module integrating Qdrant and BM25 for a local RAG pipeline.
Read more
DetailsBenchmarks: token accuracy increased from 50.67% to 74.85%.
Hardware: NVIDIA RTX 4070 Laptop (8188 MiB VRAM), local Ollama and Transformers bfloat16.
Training: 3 epochs, 507 SFT steps on 3,000 instruction pairs.










Comments