# PicoSAM3: Real-Time In-Sensor Region-of-Interest Segmentation

arXiv: https://arxiv.org/abs/2603.11917 · HTML: https://mlsanity.com/paper/2603.11917/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 57 references that are in the map.
- First posted on arXiv 12 Mar 2026.

## Builds on (most cited first, 30 of 34)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [SAM 3: Segment Anything with Concepts](https://mlsanity.com/paper/2511.16719/) (2025)
- [A Survey of Quantization Methods for Efficient Neural Network Inference](https://mlsanity.com/paper/2103.13630/) (2021)
- [Up or Down? Adaptive Rounding for Post-Training Quantization](https://mlsanity.com/paper/2004.10568/) (2020)
- [ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks](https://mlsanity.com/paper/1910.03151/) (2019)
- [Faster Segment Anything: Towards Lightweight SAM for Mobile Applications](https://mlsanity.com/paper/2306.14289/) (2023)
- [Hiera: A Hierarchical Vision Transformer without the Bells-and-Whistles](https://mlsanity.com/paper/2306.00989/) (2023)
- [Fast Segment Anything](https://mlsanity.com/paper/2306.12156/) (2023)
- [Efficient Deep Learning: A Survey on Making Deep Learning Models Smaller, Faster, and Better](https://mlsanity.com/paper/2106.08962/) (2021)
- [MobileNetV4 -- Universal Models for the Mobile Ecosystem](https://mlsanity.com/paper/2404.10518/) (2024)
- [A Survey on Segment Anything Model (SAM): Vision Foundation Model Meets Prompt Engineering](https://mlsanity.com/paper/2306.06211/) (2023)
- [MobileSAMv2: Faster Segment Anything to Everything](https://mlsanity.com/paper/2312.09579/) (2023)
- [Expediting Large-Scale Vision Transformer for Dense Prediction without Fine-tuning](https://mlsanity.com/paper/2210.01035/) (2022)
- [TinySAM: Pushing the Envelope for Efficient Segment Anything Model](https://mlsanity.com/paper/2312.13789/) (2023)
- [PTQ4SAM: Post-Training Quantization for Segment Anything](https://mlsanity.com/paper/2405.03144/) (2024)
- [SlimSAM: 0.1% Data Makes Segment Anything Slim](https://mlsanity.com/paper/2312.05284/) (2023)
- [FANN-on-MCU: An Open-Source Toolkit for Energy-Efficient Neural Network Inference at the Edge of the Internet of Things](https://mlsanity.com/paper/1911.03314/) (2019)
- [TinyTracker: Ultra-Fast and Ultra-Low-Power Edge Vision In-Sensor for Gaze Estimation](https://mlsanity.com/paper/2307.07813/) (2023)
- [Ultra-Efficient On-Device Object Detection on AI-Integrated Smart Glasses with TinyissimoYOLO](https://mlsanity.com/paper/2311.01057/) (2023)
- [HPTQ: Hardware-Friendly Post Training Quantization](https://mlsanity.com/paper/2109.09113/) (2021)
- [EPTQ: Enhanced Post-Training Quantization via Hessian-guided Network-wise Optimization](https://mlsanity.com/paper/2309.11531/) (2023)
- [PicoSAM2: Low-Latency Segmentation In-Sensor for Edge Vision Applications](https://mlsanity.com/paper/2506.18807/) (2025)

## Built on by (most cited first, 1 of 1)

- [Exploiting In-Sensor Computing for Energy-Efficient Earth Observation](https://mlsanity.com/paper/2606.01271/) (2026)

## Abstract (arXiv)

Real-time, on-device segmentation is critical for latency-sensitive and privacy-aware applications such as smart glasses and Internet-of-Things devices. We introduce PicoSAM3, a lightweight promptable visual segmentation model optimized for edge and in-sensor execution, including deployment on the Sony IMX500 vision sensor. PicoSAM3 has 1.3M parameters and combines a dense CNN architecture with region of interest prompt encoding, Efficient Channel Attention, and knowledge distillation from SAM2 and SAM3. On COCO and LVIS, PicoSAM3 achieves 65.45% and 64.01% mIoU, respectively, outperforming existing SAM-based and edge-oriented baselines at similar or lower complexity. The INT8 quantized model preserves accuracy with negligible degradation while enabling real-time in-sensor inference at 11.82ms latency on the IMX500, fully complying with its memory and operator constraints. Ablation studies show that distillation from large SAM models yields up to +14.5% mIoU improvement over supervised training and demonstrate that high-quality, spatially flexible promptable segmentation is feasible directly at the sensor level.