Qwen3.8-27B-abliterated-colibri-i8
multimodal Qwen3.8 model for the colibri engine
2 min readupdated
This build packages an abliterated (a technique that removes built-in restrictions and refusals) version of the multimodal Qwen3.8-27B model specifically for the colibrì engine. It generates text and analyzes images using a built-in 27-layer vision tower. The weights in the container are stored in uncompressed f16 format, while the engine quantizes them on the fly when loading into memory. This release targets local inference researchers and developers testing dense models on CPUs.
Key features
- Hybrid dense architecture: 64 layers where 3 out of every 4 layers use Gated DeltaNet linear attention, and every fourth layer uses full attention.
- Image support: the checkpoint includes a 27-layer vision tower (
model-vision.safetensors), allowing you to pass images as input. - Flexible load-time quantization: the
COLI_DENSE_BITSenvironment variable lets you switch between int8 mode (default, ~29 GB RAM) and selective int4 (~18–19 GB RAM). - colibrì exclusive: packaged across 66 safetensors files (~51 GiB) for the
qwen36engine, this model will not run intransformersorvLLM.
What you can do
- Run locally on CPU: deploy a 27B vision-capable model on a laptop or PC without a GPU, balancing memory usage with selective int4 quantization (
COLI_DENSE_INT4). - Run red-team tests and research: study model behavior without built-in refusals in an isolated environment, or generate synthetic data.
- Write creative fiction: draft scripts and stories without triggering the base model's standard safety filters.
Example
git clone [https://github.com/JustVugg/colibri](https://github.com/JustVugg/colibri) && cd colibri/c
make qwen36
./coli chat --model /path/to/this/repo --gpu noneRead more
DetailsLaptop speed (CPU-only): 1.6 tok/s in default int8 mode, and 2.9 tok/s with
COLI_DENSE_BITS=4 and COLI_DENSE_INT4=lmhead,dnproj,dnout,shexp.Limitations: the MTP head was intentionally stripped during conversion, so speculative decoding is unsupported.
No quality or perplexity benchmarks were run after conversion and quantization, and the original abliterated source repository is not listed in the model card.










Comments