Qwen2.5-7B GCAD steering controls
on-the-fly sarcasm and rudeness control
2 min readupdated
A set of ready-made steering banks (activation vectors that modify a model's behavior directly during generation) by RenormalizationTech for Qwen2.5-7B-Instruct. Unlike standard LoRAs or full fine-tunes, the underlying model weights stay untouched — response style is switched dynamically through the inference runtime. Three separate controllers are available: profanity, impolite tone, and sarcasm. Built for LLM steering researchers and developers who need flexible persona control for Qwen2.5-7B bots.
Key features
- Runs on top of a pinned revision of Qwen/Qwen2.5-7B-Instruct without modifying base weights or using LoRA adapters.
- Three isolated banks for distinct stylistic traits:
profanity,impolite, andsarcasm. - Includes a custom Python runtime module with SHA256 manifest integrity checks and a built-in zero-coefficient self-test.
- Open Apache-2.0 license with no extra usage restrictions.
What you can do
- Run a local dialogue generator on Qwen2.5-7B-Instruct and dial sarcasm or bluntness up and down by selecting the corresponding bank.
- Use the prebuilt banks and reports (the repo includes methodology notes and transfer results on a 4B model) for activation steering experiments.
Example
cd controls
python -m runtime.example --snapshot "$LOCAL_BASE_SNAPSHOT" --trait profanity --manifest-sha256 "$BANK_MANIFEST_SHA" --zero-selftest
python -m runtime.example --snapshot "$LOCAL_BASE_SNAPSHOT" --trait profanity --manifest-sha256 "$BANK_MANIFEST_SHA" --prompt "Describe a rainy afternoon in two sentences."
Read more
DetailsTrait expression metrics (baseline to GCAD): PF (b=5) increased from 0 to 31.13 (coherence 85.34 to 82.69); SARC (b=2) rose from 0.06 to 63.78 with coherence dropping from 89.59 to 70.72; Impolite (b=3.5) rose from 1.62 to 77.37 (coherence 82.68 to 75.9).
Requires exact base commit
a09a35458c702b33eeacc393d103063234e8bc28 (downloaded separately), Python 3.11, torch 2.6.0+cu124, and transformers 4.52.3.Controllers were calibrated individually and have not been validated for combined use; sarcasm shows a noticeable coherence drop (-18.88 points).










Comments