SonoIntent-Models
ultrasound segmentation driven by clinical questions
2 min readupdated
A collection of two fine-tuned models (VRS-HQ and Sa2VA-14B) by QingboKang built for reasoning segmentation on ultrasound images. You feed in an ultrasound scan and a clinical question, and the model outputs a text rationale alongside a binary mask of the target anatomical structure at full original resolution. Under the hood are VRS-HQ (14.58 GB, bf16) and Sa2VA-14B (30.74 GB, fp16) weights trained on the SonoIntent clinical benchmark. Unlike standard segmentation models that rely strictly on class labels, these checkpoints analyze the medical query first and locate the target region by context. This release is built for medical AI researchers looking to reproduce the paper's results or experiment with ultrasound analysis.
Key features
- Two standalone checkpoints: the repo ships ready-to-use VRS-HQ and Sa2VA-14B weights identical to those evaluated in the paper.
- Reasoning before masking: both models generate up to 384 tokens of text explanation while simultaneously predicting the binary mask of the target structure.
- Built-in visualization and metrics: included scripts automatically save 4-panel comparison grids and calculate Dice, IoU, precision, and recall.
- Flexible deployment: supports both GPU and CPU inference, letting you assign the vision and LLM components of Sa2VA-14B to separate devices.
What you can do
- Test structure segmentation (such as the fetal head or pubic symphysis) on 6 ready-to-run clinical cases inside
examples/cases. - Reproduce the SonoIntent benchmark using the public split of the
SonoIntent-Annotationsdataset (16,869 tasks across 7 open ultrasound sources). - Plug the pipeline into a research prototype for automated ultrasound annotation and analysis driven by a clinician's text prompts.
Example
# Запуск VRS-HQ на одном тестовом примере
python VRS-HQ/test.py --limit 1
# Запуск Sa2VA-14B на одном тестовом примере
python Sa2VA-14B/test.py --limit 1
Read more
DetailsFull SonoIntent test split metrics (22,870 tasks): VRS-HQ hits 50.82% Overall Dice, 40.18% TargetAcc, and 33.50% TA-Dice; Sa2VA-14B reaches 48.40% Overall Dice, 98.59% TargetAcc, and 47.51% TA-Dice.
Requirements: Python 3.10+, PyTorch 2.1+. VRS-HQ needs around 15 GB of RAM or a 24 GB VRAM GPU; Sa2VA-14B requires around 31 GB of RAM or a 40 GB GPU (or dual 24 GB cards).
On first launch, VRS-HQ pulls
openai/clip-vit-large-patch14 from Hugging Face.Limitations: for research use only; not a medical device. Weight licenses: Apache-2.0 (including Llama-2 components for VRS-HQ).










Comments