VisAlloc-9B
a multimodal model based on Qwen3.5-9B
2 min readupdated
VisAlloc-9B is a multimodal LLM (a large language model that works with both text and images) built on the Qwen/Qwen3.5-9B family and trained using the VisAlloc visual self-distillation method. It takes a full image and a text prompt as input, delivering accurate answers without cropping or zooming into image regions during inference. This makes it useful for visual question answering (VQA) and fine-grained detail recognition in images.
Key features
- Full-image inference — processes full images through a standard multimodal interface without extra cropping tools.
- Paired-view self-distillation — trained with positive and negative crops to calibrate the model's visual attention.
- Selective token supervision — selectively weights tokens based on sensitivity and teacher distribution divergence.
- Standard model release — complete set of weights in BF16 safetensors format, along with the config, tokenizer, and processor.
What you can do
- Run the model locally using the standard
transformerslibrary to answer questions about images. - Serve the model via vLLM with an OpenAI-compatible API to integrate it into third-party services and bots.
- Analyze visual content inside automated pipelines with high detail accuracy.
Example
from transformers import AutoModelForImageTextToText, AutoProcessor
model_id = "Kaelyn01/VisAlloc-9B"
model = AutoModelForImageTextToText.from_pretrained(
model_id, dtype="bfloat16", device_map="auto"
)
processor = AutoProcessor.from_pretrained(model_id)
messages = [{
"role": "user",
"content": [
{"type": "image", "url": "[https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg](https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg)"},
{"type": "text", "text": "What is the color of the label on the bottle in the background?"},
],
}]
inputs = processor.apply_chat_template(
messages,
add_generation_prompt=True,
tokenize=True,
return_dict=True,
return_tensors="pt",
).to(model.device)
outputs = model.generate(
**inputs,
max_new_tokens=512,
do_sample=False,
eos_token_id=processor.tokenizer.eos_token_id,
pad_token_id=processor.tokenizer.pad_token_id,
)
print(processor.decode(
outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True
))
Read more
DetailsBase model: Qwen/Qwen3.5-9B (9.41B parameters).
Dataset: VisAlloc-6K (5,795 curated examples).
Weights format: BF16 safetensors.
License: Apache-2.0.










Comments