JevNavAgent: A visual agent for gigapixel pathology analysis
2 min readupdated
The JevNavAgent model is a suite of pre-trained checkpoints built on the Qwen/Qwen3-VL-2B-Instruct model, supplemented with LoRA weights (a small add-on to the base model) and specialized decision-making heads. The system is designed for automated navigation and visual analysis of gigapixel medical scans (WSIs). The agent sequentially examines tissue, assesses whether to stop searching, and outputs precise answers by combining SFT (supervised fine-tuning) stages and trajectory-level reinforcement learning (GRPO). The model will be useful for researchers in biomedical engineering and digital pathology.
Key features
- Base architecture: utilizes a compact multimodal
Qwen3-VL-2B-Instructmodel with efficient LoRA adapters. - Factorized heads: includes three specialized decision heads (
heads.pt) for evaluating evidence sufficiency, simple spatial geometry analysis, and context-bilinear responses. - Two-stage training: includes weights for Stage-1 SFT trajectory alignment and Stage-2 interactive GRPO (including joint VQA+KL training and optimization for the CAMELYON16 dataset).
- Specialized benchmarks: optimized for complex clinical tasks on the WSI-VQA 351 and CAM16 medical datasets.
What you can do
- Run model evaluation: apply the official evaluation command-line package to test on WSI-VQA or CAMELYON16 datasets.
- Integrate PEFT weights: use lightweight
adapter/weights and thejev_config.jsonconfiguration file (about 73 MB) for your own experiments. - Analyze gigapixel slides: apply the model together with the developer repository to search for tumors and answer visual pathomorphology questions.
Example
git clone [https://github.com/yonghongwu/JevNavAgent.git](https://github.com/yonghongwu/JevNavAgent.git)
cd JevNavAgent && pip install -e .
jevnav-download --mode eval-slides --out ./data
jevnav-eval \
--benchmark wsivqa_351 \
--policy jevnav \
--checkpoint renyimen/JevNavAgent:stage2_traj_grpo_vqa_kl \
--backbone Qwen/Qwen3-VL-2B-Instruct \
--data-dir ./data \
--output results/jevnav_vqa.jsonl
Read more
DetailsWSI-VQA 351 performance (Stage-2): Step-$T$ accuracy 50.43%, Micro-Focal 51.39%, Clin. A–D 54.73%.
Results on CAM16 Test-129: localization hit rate 22.45% (with stop conversion 91.7%), specificity 93.75%.
Each checkpoint contains PEFT LoRA weights, factorized
heads.pt heads, and a jev_config.json file (~73 MB).License: Apache-2.0 (the base model is subject to the original Qwen license).










Comments