# PolSAM: Polarimetric Scattering Mechanism Informed Segment Anything Model

arXiv: https://arxiv.org/abs/2412.12737 · HTML: https://mlsanity.com/paper/2412.12737/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 54 references that are in the map.
- First posted on arXiv 17 Dec 2024.

## Builds on (most cited first, 14 of 14)

- [Medical SAM Adapter: Adapting Segment Anything Model for Medical Image Segmentation](https://mlsanity.com/paper/2304.12620/) (2023)
- [Faster Segment Anything: Towards Lightweight SAM for Mobile Applications](https://mlsanity.com/paper/2306.14289/) (2023)
- [Personalize Segment Anything Model with One Shot](https://mlsanity.com/paper/2305.03048/) (2023)
- [CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation with Transformers](https://mlsanity.com/paper/2203.04838/) (2022)
- [RSPrompter: Learning to Prompt for Remote Sensing Instance Segmentation based on Visual Foundation Model](https://mlsanity.com/paper/2306.16269/) (2023)
- [SurgicalSAM: Efficient Class Promptable Surgical Instrument Segmentation](https://mlsanity.com/paper/2308.08746/) (2023)
- [SAM-Assisted Remote Sensing Imagery Semantic Segmentation with Object and Boundary Constraints](https://mlsanity.com/paper/2312.02464/) (2023)
- [Ladder Fine-tuning approach for SAM integrating complementary network](https://mlsanity.com/paper/2306.12737/) (2023)
- [Weakly Supervised Semantic Segmentation via Alternative Self-Dual Teaching](https://mlsanity.com/paper/2112.09459/) (2021)
- [PA-SAM: Prompt Adapter SAM for High-Quality Image Segmentation](https://mlsanity.com/paper/2401.13051/) (2024)
- [Polarimetric SAR Image Semantic Segmentation with 3D Discrete Wavelet Transform and Markov Random Field](https://mlsanity.com/paper/2008.11014/) (2020)
- [Polarimetric Convolutional Network for PolSAR Image Classification](https://mlsanity.com/paper/1807.02975/) (2018)
- [Unsupervised Classification of PolSAR Data Using a Scattering Similarity Measure Derived from a Geodesic Distance](https://mlsanity.com/paper/1712.00427/) (2017)
- [Tuning a SAM-Based Model with Multi-Cognitive Visual Adapter to Remote Sensing Instance Segmentation](https://mlsanity.com/paper/2408.08576/) (2024)

## Abstract (arXiv)

PolSAR data presents unique challenges due to its rich and complex characteristics. Existing data representations, such as complex-valued data, polarimetric features, and amplitude images, are widely used. However, these formats often face issues related to usability, interpretability, and data integrity. Most feature extraction networks for PolSAR are small, limiting their ability to capture features effectively. To address these issues, We propose the Polarimetric Scattering Mechanism-Informed SAM (PolSAM), an enhanced Segment Anything Model (SAM) that integrates domain-specific scattering characteristics and a novel prompt generation strategy. PolSAM introduces Microwave Vision Data (MVD), a lightweight and interpretable data representation derived from polarimetric decomposition and semantic correlations. We propose two key components: the Feature-Level Fusion Prompt (FFP), which fuses visual tokens from pseudo-colored SAR images and MVD to address modality incompatibility in the frozen SAM encoder, and the Semantic-Level Fusion Prompt (SFP), which refines sparse and dense segmentation prompts using semantic information. Experimental results on the PhySAR-Seg datasets demonstrate that PolSAM significantly outperforms existing SAM-based and multimodal fusion models, improving segmentation accuracy, reducing data storage, and accelerating inference time. The source code and datasets will be made publicly available at https://github.com/XAI4SAR/PolSAM.