Vulnerability Detector SFT
adapters for finding C/C++ vulnerabilities
2 min readupdated
A set of three QLoRA adapters (lightweight add-ons for 4-bit models) that inspect C and C++ functions and classify them as either safe (CLEAN) or vulnerable along with the weakness type (VULNERABLE: CWE-). Developed for a master's thesis at the University of Parma, the adapters were trained using SFT (supervised fine-tuning) on the BigVul real-world vulnerability dataset. The repository includes versions for DeepSeek-Coder-6.7B-Instruct, Mistral-7B-Instruct-v0.3, and Qwen2.5-Coder-7B-Instruct. The model only flags issues without generating fixes, making it useful for developers and security researchers running initial code audits or experimenting in this space.
Key features
- Three ready-to-use QLoRA adapters (rank 32, alpha 64) organized in subfolders for popular 6.7B–7B coding and general-purpose LLMs.
- The
deepseek-coder-6.7bvariant performs best, catching 83.43% of vulnerable functions on the BigVul test split with a false positive rate under 5% (F1 = 0.5813, 7x higher than the base model without fine-tuning). - The author openly notes a key limitation: on the paired PrimeVul benchmark, where a vulnerable function is pitted directly against its patched version, the adapters perform no better than random guessing—meaning the high BigVul scores rely heavily on surface-level patterns.
- Strict output format: the model responds with a single line, omitting explanations or patching advice.
What you can do
- Wire the
deepseek-coder-6.7badapter into a local script for preliminary static analysis of C/C++ functions using 4-bit quantization. - Reproduce the thesis experiments on CWE detection and benchmark coding LLMs (DeepSeek-Coder and Qwen2.5-Coder) against the general-purpose Mistral-7B on the same pipeline.
Example
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel
bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype=torch.bfloat16)
tok = AutoTokenizer.from_pretrained("phazertron/vulnerability-detector-sft", subfolder="deepseek-coder-6.7b")
base = AutoModelForCausalLM.from_pretrained("deepseek-ai/deepseek-coder-6.7b-instruct", quantization_config=bnb, device_map="auto")
model = PeftModel.from_pretrained(base, "phazertron/vulnerability-detector-sft", subfolder="deepseek-coder-6.7b")
Read more
DetailsBigVul test split metrics:
deepseek-coder-6.7b — F1 0.5813, Recall 0.8343, FPR 0.0469; mistral-7b — F1 0.4923, Recall 0.7130, FPR 0.0536; qwen2.5-coder-7b — F1 0.2156, Recall 0.1400, FPR 0.0072.Requires a strict system prompt:
You are a security auditor. Analyze the following C function. Output ONLY: VULNERABLE: CWE- OR CLEAN; expected output looks like VULNERABLE: CWE-787 or CLEAN.Trained with PEFT 0.13.2, TRL 1.5.0, and Transformers 4.57.6 on the BigVul dataset (CASTLE and PrimeVul were excluded from training).
Mixed licensing: Apache 2.0 for the Mistral and Qwen2.5-Coder adapters, and the DeepSeek Model License for DeepSeek-Coder.










Comments