mlsanityData to 7 Oct 2026

A Comprehensive Survey on Segment Anything Model for Vision and Beyond

arXiv 2305.08196 ↗First posted 14 May 2023cs.CV cs.AIcited by 64

Not on the map. Why?

Abstract (arXiv)

Artificial intelligence (AI) is evolving towards artificial general intelligence, which refers to the ability of an AI system to perform a wide range of tasks and exhibit a level of intelligence similar to that of a human being. This is in contrast to narrow or specialized AI, which is designed to perform specific tasks with a high degree of efficiency. Therefore, it is urgent to design a general class of models, which we term foundation models, trained on broad data that can be adapted to various downstream tasks. The recently proposed segment anything model (SAM) has made significant progress in breaking the boundaries of segmentation, greatly promoting the development of foundation models for computer vision. To fully comprehend SAM, we conduct a survey study. As the first to comprehensively review the progress of segmenting anything task for vision and beyond based on the foundation model of SAM, this work focuses on its applications to various tasks and data types by discussing its historical development, recent progress, and profound impact on broad applications. We first introduce the background and terminology for foundation models including SAM, as well as state-of-the-art methods contemporaneous with SAM that are significant for segmenting anything task. Then, we analyze and summarize the advantages and limitations of SAM across various image processing applications, including software scenes, real-world scenes, and complex scenes. Importantly, many insights are drawn to guide future research to develop more versatile foundation models and improve the architecture of SAM. We also summarize massive other amazing applications of SAM in vision and beyond. Finally, we maintain a continuously updated paper list and an open-source project summary for foundation model SAM at hrefhttps://github.com/liliu-avril/Awesome-Segment-Anythingcolormagentahere.

Key facts. Cited by 64 papers in the map (data through 7 Oct 2026). Builds on 168 of its 223 references that are in the map. First posted on arXiv 14 May 2023.

Builds on

168 of 223 references.

References · 168

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  6. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  7. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  8. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  9. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  10. 2017Graph Attention Networkscited by 8,854
  11. 2023Segment Anythingcited by 8,158
  12. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
38 more
  1. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  2. 2017Mask R-CNNcited by 7,259
  3. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  4. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  5. 2017Inductive Representation Learning on Large Graphscited by 6,284
  6. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  7. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  8. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  9. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  10. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  11. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,239
  12. 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,210
  13. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,923
  14. 2023A Survey of Large Language Modelscited by 2,537
  15. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,338
  16. 2016V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentationcited by 2,284
  17. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  18. 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,154
  19. 2021TransUNet: Transformers Make Strong Encoders for Medical Image Segmentationcited by 1,552
  20. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,447
  21. 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Modelscited by 1,419
  22. 2015Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Modelscited by 1,352
  23. 2021Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentationcited by 1,218
  24. 2021Swin Transformer V2: Scaling Up Capacity and Resolutioncited by 1,215
  25. 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,162
  26. 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,148
  27. 2021YOLOX: Exceeding YOLO Series in 2021cited by 1,064
  28. 2016Semantic Understanding of Scenes through the ADE20K Datasetcited by 1,063
  29. 2021Scaling Vision Transformerscited by 993
  30. 2019VL-BERT: Pre-training of Generic Visual-Linguistic Representationscited by 954
  31. 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 951
  32. 2020NeRF++: Analyzing and Improving Neural Radiance Fieldscited by 857
  33. 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
  34. 2021ByteTrack: Multi-Object Tracking by Associating Every Detection Boxcited by 812
  35. 2021Florence: A New Foundation Model for Computer Visioncited by 802
  36. 2018Panoptic Segmentationcited by 796
  37. 2018Path Aggregation Network for Instance Segmentationcited by 796
  38. 2021Resolution-robust Large Mask Inpainting with Fourier Convolutionscited by 730

50 most cited of 168 All 168 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 16; 2024: 23; 2025: 20; 2026: 5.

64 papers.

Sort:

By area:

  1. 2023Multimodal Foundation Models: From Specialists to General-Purpose Assistantscited by 247
  2. 2023Foundational Models Defining a New Era in Vision: A Survey and Outlookcited by 109
  3. 2023Review of Large Vision Models and Visual Prompt Engineeringcited by 77
  4. 2024ClipSAM: CLIP and SAM Collaboration for Zero-Shot Anomaly Segmentationcited by 49
  5. 2023SAMAug: Point Prompt Augmentation for Segment Anything Modelcited by 38
  6. 2024Image Segmentation in Foundation Model Era: A Surveycited by 35
  7. 2024GoodSAM: Bridging Domain and Capacity Gaps via Segment Anything Model for Distortion-aware Panoramic Semantic Segmentationcited by 27
  8. 2023Detect Any Deepfakes: Segment Anything Meets Face Forgery Detection and Localizationcited by 20
  9. 2023AutoProSAM: Automated Prompting SAM for 3D Multi-Organ Segmentationcited by 19
  10. 2024Interactive 3D Medical Image Segmentation with SAM 2cited by 17
  11. 2024On Efficient Variants of Segment Anything Model: A Surveycited by 16
  12. 2023Segment Anything Model is a Good Teacher for Local Feature Learningcited by 15
38 more
  1. 2024WebUOT-1M: Advancing Deep Underwater Object Tracking with A Million-Scale Benchmarkcited by 15
  2. 2023SAD: Segment Any RGBDcited by 14
  3. 2023SAM-OCTA: A Fine-Tuning Strategy for Applying Foundation Model to OCTA Image Segmentation Taskscited by 14
  4. 2023Tree-GPT: Modular Large Language Model Expert System for Forest Remote Sensing Image Understanding and Interactive Analysiscited by 13
  5. 2024See Further for Parameter Efficient Fine-tuning by Standing on the Shoulders of Decompositioncited by 13
  6. 2025SAM2 for Image and Video Segmentation: A Comprehensive Surveycited by 13
  7. 2024GoodSAM++: Bridging Domain and Capacity Gaps via Segment Anything Model for Panoramic Semantic Segmentationcited by 11
  8. 2025Diffusion-empowered AutoPrompt MedSAMcited by 10
  9. 2023Temporally-Extended Prompts Optimization for SAM in Interactive Medical Image Segmentationcited by 9
  10. 2025WeakMedSAM: Weakly-Supervised Medical Image Segmentation via SAM with Sub-Class Exploration and Prompt Affinity Miningcited by 8
  11. 2023Large Scale Foundation Models for Intelligent Manufacturing Applications: A Surveycited by 6
  12. 2023SAM-OCTA: Prompting Segment-Anything for OCTA Image Segmentationcited by 5
  13. 2024Segment Anything for Videos: A Systematic Surveycited by 5
  14. 2024Underwater Camouflaged Object Tracking Meets Vision-Language SAM2cited by 5
  15. 2024Real-Time Localization and Bimodal Point Pattern Analysis of Palms Using UAV Imagerycited by 5
  16. 2024MambaTrack: Exploiting Dual-Enhancement for Night UAV Trackingcited by 5
  17. 2023Industrial Segment Anything -- a Case Study in Aircraft Manufacturing, Intralogistics, Maintenance, Repair, and Overhaulcited by 4
  18. 2023Promoting Segment Anything Model towards Highly Accurate Dichotomous Image Segmentationcited by 4
  19. 2024Strike the Balance: On-the-Fly Uncertainty based User Interactions for Long-Term Video Object Segmentationcited by 4
  20. 2024Multi-scale Contrastive Adaptor Learning for Segmenting Anything in Underperformed Scenescited by 4
  21. 2025UncertainSAM: Fast and Efficient Uncertainty Quantification of the Segment Anything Modelcited by 4
  22. 2023Masking Hyperspectral Imaging Data with Pretrained Modelscited by 3
  23. 2024A Literature Review of Literature Reviews in Pattern Analysis and Machine Intelligencecited by 3
  24. 2025SAM-DA: Decoder Adapter for Efficient Medical Domain Adaptationcited by 3
  25. 2025Zero-shot Domain Generalization of Foundational Models for 3D Medical Image Segmentation: An Experimental Studycited by 3
  26. 2025SAM-aware Test-time Adaptation for Universal Medical Image Segmentationcited by 3
  27. 2025Prompt Engineering in Segment Anything Model: Methodologies, Applications, and Emerging Challengescited by 3
  28. 2024SAMEdge: An Edge-cloud Video Analytics Architecture for the Segment Anything Modelcited by 2
  29. 2024New Paradigm of Adversarial Training: Releasing Accuracy-Robustness Trade-Off via Dummy Classcited by 2
  30. 2025COST: Contrastive One-Stage Transformer for Vision-Language Small Object Trackingcited by 2
  31. 2025DOMR: Establishing Cross-View Segmentation via Dense Object Matchingcited by 2
  32. 2025Easy3D-Labels: Supervising Semantic Occupancy Estimation with 3D Pseudo-Labels for Automotive Perceptioncited by 2
  33. 2026On The Robustness of Foundational 3D Medical Image Segmentation Models Against Imprecise Visual Promptscited by 2
  34. 2026SegMoTE: Token-Level Mixture of Experts for Medical Image Segmentationcited by 2
  35. 2026SSP-SAM: SAM with Semantic-Spatial Prompt for Referring Expression Segmentationcited by 2
  36. 2024ClassWise-SAM-Adapter: Parameter Efficient Fine-tuning Adapts Segment Anything to SAR Domain for Semantic Segmentationcited by 1
  37. 2024Opportunities and challenges in the application of large artificial intelligence models in radiologycited by 1
  38. 2024Urban Waterlogging Detection: A Challenging Benchmark and Large-Small Model Co-Adaptercited by 1

50 most cited of 64 All 64 (JSON)