Vulnerability Detector
finding vulnerabilities in C code and LLVM IR
2 min readupdated
This repository contains two QLoRA adapters (lightweight add-ons on top of a base model using 4-bit quantization) built on Qwen2.5-Coder-7B-Instruct by phazertron as part of a master's thesis at the University of Parma. Both variants are trained on the synthetic SARD Juliet dataset to detect vulnerabilities and classify them by CWE (Common Weakness Enumeration — a standard catalog of software weakness types): the first analyzes C source code, while the second works on LLVM IR intermediate representation. The model outputs a single-line verdict (such as VULNERABLE: CWE-787 or CLEAN). The main finding of the experiment is that the LLVM IR variant is worse at detecting whether a bug is present at all, yet substantially more accurate at naming the specific CWE category. This release is useful for security researchers and ML engineers looking to reproduce the thesis experiments, though it is not meant for auditing real-world codebases.
Key features
- Two adapters in one repo: the
irandsourcesubfolders hold the variants for LLVM IR (compiled viaclang -O0 -emit-llvm) and C source code, respectively. - Contrasting metrics: on the test split, the
irvariant reached 0.7742 CWE classification accuracy versus 0.1455 forsource(whilesourceleads in Binary F1 at 0.5069 against 0.3116). - Shared base and setup: both adapters are trained on synthetic SARD Juliet C programs on top of Qwen2.5-Coder-7B-Instruct under the Apache 2.0 license.
What you can do
- Reproduce the research experiment: compare how an LLM behaves when analyzing raw source code versus low-level LLVM IR for the exact same functions.
- Explore vulnerability attribution: test classification across 13 distinct CWE categories using synthetic examples from the SARD Juliet suite.
Example
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel
base_id = "Qwen/Qwen2.5-Coder-7B-Instruct"
repo = "phazertron/vulnerability-detector-ir-source"
bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype=torch.bfloat16)
model = PeftModel.from_pretrained(AutoModelForCausalLM.from_pretrained(base_id, quantization_config=bnb, device_map="auto"), repo, subfolder="ir")Read more
DetailsMetrics (246 samples): Binary F1 — 0.5069 (source) and 0.3116 (ir); CWE accuracy when detected — 0.1455 (source) and 0.7742 (ir).
QLoRA settings: rank 32, alpha 64, dropout 0.05, 2048-token context budget, 4-bit NF4.
Stack: PEFT 0.13.2, TRL 1.5.0, Transformers 4.57.6.
Limitations: trained on synthetic SARD Juliet data and does not generalize to real production code.










Comments