MobileSAM-ONNX: Accelerated ONNX graphs for image segmentation
2 min readupdated
A set of weights and graphs in the ONNX format for MobileSAM v2 (a lightweight version of the Segment Anything image segmentation model), prepared by the moonbox-dev team for running on devices using ONNX Runtime (CPU, CUDA, DirectML). The model consists of a compact TinyViT-5M image encoder and a mask decoder, replacing the heavy components of the original for high performance on mobile and local devices. The solution comes with a ready-to-use pipeline.py file for image preprocessing and prompt coordinate processing. This new release is useful for developers of mobile apps, local services, and automated data labeling systems.
Key features
- Lightweight TinyViT-5M encoder (just 9.8M parameters in total)
- ONNX Runtime support for CPU, CUDA, and DirectML
- fp16 precision with minimal mask deviation by the IoU metric
- Built-in
pipeline.pyscript for scaling, normalization, and working with points/boxes
What you can do
- Integrate the segmentation model into mobile or local desktop apps via ONNX Runtime
- Quickly generate object masks from points or bounding boxes on user photos
Example
import sys, numpy as np
from PIL import Image
sys.path.insert(0, "fp16")
import pipeline
sam = pipeline.MobileSAM("fp16")
emb, size = sam.embed(np.array(Image.open("photo.jpg").convert("RGB")))
masks, scores = sam.point(emb, size, [[410, 300]]) # 3 masks [3, H, W] (bool)
box_mask, score = sam.box(emb, size, [120, 80, 700, 560])Read more
DetailsBenchmark: average mask IoU 0.998 / worst 0.977 for the fp16 build
File size: encoder 14.1 MB, decoder 16.5 MB
Dependencies: numpy, onnxruntime (or onnxruntime-gpu), Pillow
License: apache-2.0










Comments