gemma_3_4b_it-vlguard-sft-fable-report-set1-refusal-neurons-zeroed
Gemma 3 4B with refusal neurons zeroed out
2 min readupdated
A 4.3B-parameter research checkpoint of a multimodal model (handling both text and images) based on Adeely93/gemma_3_4b_it-vlguard-sft-epoch1.5-lr1e-6, created by Darth-Coder. As part of the WSP-MLLM-Safety study, the model was modified without fine-tuning: guided by an interpretability map, 1,000 specific MLP neurons responsible for refusing unsafe requests were zeroed out across decoder layers 12–29. Unlike standard jailbreak fine-tunes, this surgical edit alters only 0.18% of the weights (7.68 million parameters), leaving general capabilities virtually intact. The artifact is intended for AI safety, alignment, and red-teaming researchers.
Key features
- Surgical weight modification: only 1,000 "harm→refusal" neurons in the
mlp.down_proj.weightmatrices across layers 12–29 are zeroed out, with no additional training. - Sharp drop in refusal rates: on multimodal safety benchmarks, refusal rates fall from 73–100% down to 2–32%.
- Preserved base capabilities: accuracy on standard tasks like ScienceQA and MathVista drops by just 2–5 percentage points.
- Strictly for research: the model card notes that this is a red-team artifact not intended for production or general distribution.
What you can do
- Study multimodal alignment robustness: test how easily defenses trained via VLGuard-SFT can be bypassed using mechanistic interpretability methods.
- Reproduce neuron ablation experiments: compare the behavior of the zeroed-out model against the original SFT checkpoint on your own multimodal prompt sets.
- Develop defenses against weight editing: use the checkpoint as a reference case for vulnerabilities to targeted removal of refusal mechanisms.
Example
A scenario from the study compares the original SFT model and the zeroed-neuron version on the FigStep multimodal attack set (100 samples, greedy decoding): the baseline model refuses 100% of the time (0% attack success rate by keyword matching), while the modified version drops the refusal rate to 7% and generates a coherent, non-refusal response in 92% of cases.
Read more
DetailsSafety benchmarks (SFT → edited): FigStep refusal 100% → 7% (ASR proxy 0% → 92%), MM-SafetyBench 73% → 2% (ASR proxy 27% → 96%), JB-Mini 72% → 32% (ASR proxy 27% → 66%).
Utility (SFT → edited): ScienceQA 76% → 74%, MathVista 52% → 47%.
Details: 7.68M parameters affected (0.18% of 4.3B); the ASR metric in the table is a proxy based on the absence of refusal keywords rather than the official gpt-4.1-mini judge.
Restrictions: distributed under the Gemma Terms of Use and Gemma Prohibited Use Policy for safety research only.










Comments