AI-SAM: Automatic and Interactive Segment Anything Model
Not on the map. Why?
Abstract (arXiv)
Semantic segmentation is a core task in computer vision. Existing methods are generally divided into two categories: automatic and interactive. Interactive approaches, exemplified by the Segment Anything Model (SAM), have shown promise as pre-trained models. However, current adaptation strategies for these models tend to lean towards either automatic or interactive approaches. Interactive methods depend on prompts user input to operate, while automatic ones bypass the interactive promptability entirely. Addressing these limitations, we introduce a novel paradigm and its first model: the Automatic and Interactive Segment Anything Model (AI-SAM). In this paradigm, we conduct a comprehensive analysis of prompt quality and introduce the pioneering Automatic and Interactive Prompter (AI-Prompter) that automatically generates initial point prompts while accepting additional user inputs. Our experimental results demonstrate AI-SAM's effectiveness in the automatic setting, achieving state-of-the-art performance. Significantly, it offers the flexibility to incorporate additional user prompts, thereby further enhancing its performance. The project page is available at https://github.com/ymp5078/AI-SAM.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 54 of its 69 references that are in the map. First posted on arXiv 5 Dec 2023.
Builds on
54 of 69 references.
References · 54
- 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
- 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
- 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
- 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,239
- 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,210
- 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,923
- 2016Pyramid Scene Parsing Networkcited by 2,472
- 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,338
- 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,154
- 2021TransUNet: Transformers Make Strong Encoders for Medical Image Segmentationcited by 1,552
- 2015SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentationcited by 1,483
- 2020Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformerscited by 1,368
38 more
- 2021Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentationcited by 1,218
- 2018Dual Attention Network for Scene Segmentationcited by 1,001
- 2021Segmenter: Transformer for Semantic Segmentationcited by 855
- 2022EVA: Exploring the Limits of Masked Visual Representation Learning at Scalecited by 685
- 2018BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentationcited by 645
- 2022Vision Transformer Adapter for Dense Predictionscited by 564
- 2014Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFscited by 552
- 2023Medical SAM Adapter: Adapting Segment Anything Model for Medical Image Segmentationcited by 485
- 2022InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutionscited by 479
- 2020Asymmetric Loss For Multi-Label Classificationcited by 379
- 2023Segment Anything in High Qualitycited by 373
- 2013DeepPose: Human Pose Estimation via Deep Neural Networkscited by 305
- 2023Customized Segment Anything Model for Medical Image Segmentationcited by 277
- 2017Integral Human Pose Regressioncited by 255
- 2021Anabranch Network for Camouflaged Object Segmentationcited by 227
- 2019Asymmetric Non-local Neural Networks for Semantic Segmentationcited by 196
- 2023Segment Anything Model (SAM) for Digital Pathology: Assess Zero-shot Segmentation on Whole Slide Imagingcited by 161
- 2017Stacked Conditional Generative Adversarial Networks for Jointly Learning Shadow Detection and Shadow Removalcited by 160
- 2021Camouflaged Object Segmentation with Distraction Miningcited by 151
- 2014Joint Training of a Convolutional Network and a Graphical Model for Human Pose Estimationcited by 129
- 2021Query2Label: A Simple Transformer Way to Multi-Label Classificationcited by 126
- 2021Rethinking BiSeNet For Real-time Semantic Segmentationcited by 124
- 2023ONE-PEACE: Exploring One General Representation Model Toward Unlimited Modalitiescited by 117
- 2021Uncertainty-aware Joint Salient Object and Camouflaged Object Detectioncited by 114
- 2022SimpleClick: Interactive Image Segmentation with Simple Vision Transformerscited by 109
- 2021MISSFormer: An Effective Medical Image Segmentation Transformercited by 103
- 2021Mixed Transformer U-Net For Medical Image Segmentationcited by 95
- 2023Expressive Text-to-Image Generation with Rich Textcited by 89
- 2023MA-SAM: Modality-agnostic SAM Adaptation for 3D Medical Image Segmentationcited by 82
- 2019ACFNet: Attentional Class Feature Network for Semantic Segmentationcited by 75
- 2023AutoSAM: Adapting SAM to Medical Images by Overloading the Prompt Encodercited by 70
- 2023Large AI Models in Health Informatics: Applications, Challenges, and the Futurecited by 61
- 2017Direction-aware Spatial Context Features for Shadow Detectioncited by 56
- 2023How to Efficiently Adapt Large Segmentation Model(SAM) to Medical Imagescited by 53
- 2021Learning Statistical Texture for Semantic Segmentationcited by 50
- 2023SAMAug: Point Prompt Augmentation for Segment Anything Modelcited by 38
- 2023AdaptiveSAM: Towards Efficient Tuning of SAM for Surgical Scene Segmentationcited by 37
- 2023Multi-scale Hierarchical Vision Transformer with Cascaded Attention Decoding for Medical Image Segmentationcited by 34
50 most cited of 54 All 54 (JSON)
Built on by
Citing papers per year: 2025: 2.
2 papers.