mlsanityData to 14 Oct 2026

SAQ-SAM: Semantically-Aligned Quantization for Segment Anything Model

arXiv 2503.06515 ↗First posted 9 Mar 2025cs.CVcited by 3

Not on the map. Why?

Abstract (arXiv)

Segment Anything Model (SAM) exhibits remarkable zero-shot segmentation capability; however, its prohibitive computational costs make edge deployment challenging. Although post-training quantization (PTQ) offers a promising compression solution, existing methods yield unsatisfactory results when applied to SAM, owing to its specialized model components and promptable workflow: (i) The mask decoder's attention exhibits extreme activation outliers, and we find that aggressive clipping (even 100x), without smoothing or isolation, is effective in suppressing outliers while maintaining performance. Unfortunately, traditional distribution-based metrics (e.g., MSE) fail to provide such large-scale clipping. (ii) Existing quantization reconstruction methods neglect semantic interactivity of SAM, leading to misalignment between image feature and prompt intention. To address the above issues, we propose SAQ-SAM in this paper, which boosts PTQ for SAM from the perspective of semantic alignment. Specifically, we propose Perceptual-Consistency Clipping, which exploits attention focus overlap to promote aggressive clipping while preserving semantic capabilities. Furthermore, we propose Prompt-Aware Reconstruction, which incorporates image-prompt interactions by leveraging cross-attention in mask decoder, thus facilitating alignment in both distribution and semantic. Moreover, to ensure the interaction efficiency, we design a layer-skipping strategy for image tokens in encoder. Extensive experiments are conducted on various SAM sizes and tasks, including instance segmentation, oriented object detection, and semantic segmentation, and the results show that our method consistently exhibits advantages. For example, when quantizing SAM-B to 4-bit, SAQ-SAM achieves 11.7% higher mAP than the baseline in instance segmentation task.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 42 of its 54 references that are in the map. First posted on arXiv 9 Mar 2025.

Builds on

42 of 54 references.

References · 42

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  4. 2023Segment Anythingcited by 8,234
  5. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,940
  6. 2024SAM 2: Segment Anything in Images and Videoscited by 2,863
  7. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,593
  8. 2019FCOS: Fully Convolutional One-Stage Object Detectioncited by 1,577
  9. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  10. 2022DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detectioncited by 1,319
  11. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  12. 2021YOLOX: Exceeding YOLO Series in 2021cited by 1,071
30 more
  1. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 815
  2. 2022Exploring Plain Vision Transformer Backbones for Object Detectioncited by 724
  3. 2019Learned Step Size Quantizationcited by 628
  4. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 621
  5. 2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 479
  6. 2018PACT: Parameterized Clipping Activation for Quantized Neural Networkscited by 467
  7. 2023OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Modelscited by 424
  8. 2023Faster Segment Anything: Towards Lightweight SAM for Mobile Applicationscited by 352
  9. 2019Data-Free Quantization Through Weight Equalization and Bias Correctioncited by 336
  10. 2023Fast Segment Anythingcited by 290
  11. 2021Post-Training Quantization for Vision Transformercited by 278
  12. 2021Object Detection in Aerial Images: A Large-Scale Benchmark and Challengescited by 204
  13. 2023EfficientSAM: Leveraged Masked Image Pretraining for Efficient Segment Anythingcited by 188
  14. 2023RSPrompter: Learning to Prompt for Remote Sensing Instance Segmentation based on Visual Foundation Modelcited by 181
  15. 2022QDrop: Randomly Dropping Quantization for Extremely Low-bit Post-Training Quantizationcited by 180
  16. 2022DETRs with Hybrid Matchingcited by 150
  17. 2022RepQ-ViT: Scale Reparameterization for Post-Training Quantization of Vision Transformerscited by 137
  18. 2022I-ViT: Integer-only Quantization for Efficient Vision Transformer Inferencecited by 95
  19. 2022PD-Quant: Post-Training Quantization based on Prediction Difference Metriccited by 73
  20. 2022Patch Similarity Aware Data-Free Quantization for Vision Transformerscited by 56
  21. 2022PSAQ-ViT V2: Towards Accurate and General Data-Free Quantization for Vision Transformerscited by 42
  22. 2024PTQ4SAM: Post-Training Quantization for Segment Anythingcited by 32
  23. 2024EDA-DM: Enhanced Distribution Alignment for Post-Training Quantization of Diffusion Modelscited by 23
  24. 2023SlimSAM: 0.1% Data Makes Segment Anything Slimcited by 20
  25. 2023QFT: Quantized Full-parameter Tuning of LLMs with Affordable Resourcescited by 10
  26. 2024RepQuant: Towards Accurate Post-Training Quantization of Large Transformer Models via Scale Reparameterizationcited by 10
  27. 2023BinaryViT: Towards Efficient and Accurate Binary Vision Transformerscited by 7
  28. 2024A SAM-guided Two-stream Lightweight Model for Anomaly Detectioncited by 7
  29. 2024Privacy-Preserving SAM Quantization for Efficient Edge Intelligence in Healthcarecited by 4
  30. 2024Interactions Across Blocks in Post-Training Quantization of Large Language Modelscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026PTQ4ARVG: Post-Training Quantization for AutoRegressive Visual Generation Modelscited by 4
  2. 2026Efficient-SAM2: Accelerating SAM2 with Object-Aware Visual Encoding and Memory Retrievalcited by 2
  3. 2026Lean-SAM2: Target-Anchored Memory and Encoder Acceleration for SAM2cited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025AHCQ-SAM: Toward Accurate and Hardware-Compatible Post-Training Segment Anything Model Quantization
  2. 2026CAR-SAM: Cross-Attention Reconstruction for Post-Training Quantization of the Segment Anything Model
  3. 2025Q-SAM2: Accurate Quantization for Segment Anything Model 2
  4. 2025Mix-QSAM: Mixed-Precision Quantization of the Segment Anything Model
  5. 2026CLIP-Guided SAM: Parameter-Efficient Semantic Conditioning for Promptable Segmentation
  6. 2023TinySAM: Pushing the Envelope for Efficient Segment Anything Model
  7. 2025Vision and Language Reference Prompt into SAM for Few-shot Segmentation
  8. 2024PTQ4RIS: Post-Training Quantization for Referring Image Segmentation