ColQwen3.5-4.5B-v4
visual document retrieval powered by Qwen3.5-4B
2 min readupdated
ColQwen3.5-4.5B-v4 by athrael-soju tackles visual document retrieval by matching text queries directly against PDF pages, charts, and scans without running OCR first. Built on the Qwen3.5-4B architecture with bidirectional attention, the 4.54B-parameter model uses late interaction: both queries and document pages are encoded into sets of token vectors, and relevance is scored via MaxSim. Unlike the base model, it is specialized for multi-vector search and supports Matryoshka embeddings (a technique that lets you truncate vectors without a major drop in accuracy). It is well suited for developers building multimodal RAG pipelines and enterprise search engines over complex reports.
Key features
- Flexible vector dimensions: the projection layer is trained with Matryoshka prefixes at 64, 128, 256, 512, 1024, and 2048 dimensions, cutting index size by 2x to 32x.
- Multi-stage distillation: v4 is trained using a larger teacher model and a reranker that filters out false negatives, combined via LoRA adapter merging.
- Six-language support: model metadata lists support for English, French, German, Spanish, Italian, and Portuguese.
- Permissive license: released under the Apache 2.0 license and compatible with the
sentence-transformerslibrary.
What you can do
- Plug it into a multimodal RAG pipeline to retrieve answers from financial, medical, and scientific reports packed with complex tables and charts.
- Build a compact vector index for massive scan archives by truncating embeddings down to 64 or 256 dimensions to save memory.
- Rank document pages against text queries without an intermediate OCR parsing step.
Example
from sentence_transformers import MultiVectorEncoder
model = MultiVectorEncoder("athrael-soju/ColQwen3.5-4.5B-v4")
queries = ["Total outlay is maximum in which year?"]
images = ["[https://huggingface.co/datasets/sentence-transformers/example-documents/resolve/main/doc2.jpg](https://huggingface.co/datasets/sentence-transformers/example-documents/resolve/main/doc2.jpg)"]
q_emb = model.encode_query(queries)
i_emb = model.encode_document(images)
scores = model.similarity(q_emb, i_emb)Read more
DetailsBenchmarks (nDCG, 2048-d, 1792 visual tokens): ViDoRe V1 Mean — 93.09 (nDCG@5), ViDoRe V2 Mean — 69.78 (nDCG@5), ViDoRe V3 Mean — 67.14 (nDCG@10).
Software requirements: requires
transformers>=5.2.0 and sentence-transformers[image]>=6.0.0.Limitations: older versions of
transformers ignore the is_causal: false flag and produce corrupted embeddings.Weights format: BF16, 4.5B parameters.










Comments