mlsanityData to 14 Oct 2026

EfficientSAM: Leveraged Masked Image Pretraining for Efficient Segment Anything

arXiv 2312.00863 ↗First posted 1 Dec 2023cs.CVcited by 188

Not on the map. Why?

Abstract (arXiv)

Segment Anything Model (SAM) has emerged as a powerful tool for numerous vision applications. A key component that drives the impressive performance for zero-shot transfer and high versatility is a super large Transformer model trained on the extensive high-quality SA-1B dataset. While beneficial, the huge computation cost of SAM model has limited its applications to wider real-world applications. To address this limitation, we propose EfficientSAMs, light-weight SAM models that exhibits decent performance with largely reduced complexity. Our idea is based on leveraging masked image pretraining, SAMI, which learns to reconstruct features from SAM image encoder for effective visual representation learning. Further, we take SAMI-pretrained light-weight image encoders and mask decoder to build EfficientSAMs, and finetune the models on SA-1B for segment anything task. We perform evaluations on multiple vision tasks including image classification, object detection, instance segmentation, and semantic object detection, and find that our proposed pretraining method, SAMI, consistently outperforms other masked image pretraining methods. On segment anything task such as zero-shot instance segmentation, our EfficientSAMs with SAMI-pretrained lightweight image encoders perform favorably with a significant gain (e.g., ~4 AP on COCO/LVIS) over other fast SAM models.

Key facts. Cited by 188 papers in the map (data through 14 Oct 2026). Builds on 58 of its 72 references that are in the map. First posted on arXiv 1 Dec 2023.

Builds on

58 of 72 references.

References · 58

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  5. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  6. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  7. 2023Segment Anythingcited by 8,234
  8. 2020End-to-End Object Detection with Transformerscited by 7,414
  9. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  10. 2017Mask R-CNNcited by 7,277
  11. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  12. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
38 more
  1. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  2. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  3. 2020Exploring Simple Siamese Representation Learningcited by 2,866
  4. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
  5. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,351
  6. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,269
  7. 2014FitNets: Hints for Thin Deep Netscited by 1,653
  8. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,428
  9. 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,217
  10. 2021SimMIM: A Simple Framework for Masked Image Modelingcited by 1,153
  11. 2021iBOT: Image BERT Pre-Training with Online Tokenizercited by 903
  12. 2021Multiscale Vision Transformerscited by 897
  13. 2022Exploring Plain Vision Transformer Backbones for Object Detectioncited by 724
  14. 2021MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformercited by 642
  15. 2021Masked Feature Prediction for Self-Supervised Visual Pre-Trainingcited by 576
  16. 2020Dense Contrastive Learning for Self-Supervised Visual Pre-Trainingcited by 512
  17. 2021LeViT: a Vision Transformer in ConvNet's Clothing for Faster Inferencecited by 475
  18. 2020U2-Net: Going Deeper with Nested U-Structure for Salient Object Detectioncited by 412
  19. 2022Decoupled Knowledge Distillationcited by 388
  20. 2023Faster Segment Anything: Towards Lightweight SAM for Mobile Applicationscited by 352
  21. 2022BEiT v2: Masked Image Modeling with Vector-Quantized Visual Tokenizerscited by 303
  22. 2020Propagate Yourself: Exploring Pixel-Level Consistency for Unsupervised Visual Representation Learningcited by 297
  23. 2022Context Autoencoder for Self-Supervised Representation Learningcited by 297
  24. 2023Fast Segment Anythingcited by 290
  25. 2022EfficientFormer: Vision Transformers at MobileNet Speedcited by 241
  26. 2022TinyViT: Fast Pretraining Distillation for Small Vision Transformerscited by 234
  27. 2023Track Anything: Segment Anything Meets Videoscited by 234
  28. 2023Inpaint Anything: Segment Anything Meets Image Inpaintingcited by 233
  29. 2023Segment and Track Anythingcited by 193
  30. 2023EfficientViT: Memory Efficient Vision Transformer with Cascaded Group Attentioncited by 172
  31. 2023Segment Anything Model (SAM) for Digital Pathology: Assess Zero-shot Segmentation on Whole Slide Imagingcited by 161
  32. 2023Can SAM Segment Anything? When SAM Meets Camouflaged Object Detectioncited by 100
  33. 2014Salient Object Detection: A Surveycited by 98
  34. 2023Anything-3D: Towards Single-view Anything Reconstruction in the Wildcited by 92
  35. 2022MILAN: Masked Image Pretraining on Language Assisted Representationcited by 69
  36. 2022Next-ViT: Next Generation Vision Transformer for Efficient Deployment in Realistic Industrial Scenarioscited by 65
  37. 2022Bootstrapped Masked Autoencoders for Vision BERT Pretrainingcited by 60
  38. 2023Explain Any Concept: Segment Anything Meets Concept-Based Explanationcited by 37

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 8; 2024: 71; 2025: 71; 2026: 38.

188 papers.

Sort:

By area:

  1. 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 929
  2. 2023Osprey: Pixel Understanding with Visual Instruction Tuningcited by 159
  3. 2024SAMURAI: Adapting Segment Anything Model for Zero-Shot Visual Tracking with Motion-Aware Memorycited by 94
  4. 2024EVF-SAM: Early Vision-Language Fusion for Text-Prompted Segment Anything Modelcited by 65
  5. 2024Theia: Distilling Diverse Vision Foundation Models for Robot Learningcited by 61
  6. 2024SAM2-Adapter: Evaluating & Adapting Segment Anything 2 in Downstream Tasks: Camouflage, Shadow, Medical Image Segmentation, and Morecited by 60
  7. 2024ACE: All-round Creator and Editor Following Instructions via Diffusion Transformercited by 56
  8. 2025VidBot: Learning Generalizable 3D Actions from In-the-Wild 2D Human Videos for Zero-Shot Robotic Manipulationcited by 46
  9. 2023EdgeSAM: Prompt-In-the-Loop Distillation for SAMcited by 41
  10. 2024Aria Everyday Activities Datasetcited by 39
  11. 2024Learning Precise Affordances from Egocentric Videos for Robotic Manipulationcited by 34
  12. 2024CorrCLIP: Reconstructing Patch Correlations in CLIP for Open-Vocabulary Semantic Segmentationcited by 34
38 more
  1. 2023TinySAM: Pushing the Envelope for Efficient Segment Anything Modelcited by 33
  2. 2023Tokenize Anything via Promptingcited by 31
  3. 2025Rethinking End-to-End 2D to 3D Scene Segmentation in Gaussian Splattingcited by 31
  4. 2024Moving Object Segmentation: All You Need Is SAM (and Flow)cited by 30
  5. 2025Fast-FoundationStereo: Real-Time Zero-Shot Stereo Matchingcited by 28
  6. 2024CAT-SAM: Conditional Tuning for Few-Shot Adaptation of Segment Anything Modelcited by 27
  7. 2024PointSAM: Pointly-Supervised Segment Anything Model for Remote Sensing Imagescited by 24
  8. 2024Empowering Embodied Visual Tracking with Visual Foundation Models and Offline RLcited by 23
  9. 2024UN-SAM: Universal Prompt-Free Segmentation for Generalized Nuclei Imagescited by 22
  10. 2024Efficient Track Anythingcited by 21
  11. 2025EdgeTAM: On-Device Track Anything Modelcited by 21
  12. 2023SlimSAM: 0.1% Data Makes Segment Anything Slimcited by 20
  13. 2024Mamba or RWKV: Exploring High-Quality and High-Efficiency Segment Anything Modelcited by 18
  14. 2023RepViT-SAM: Towards Real-Time Segmenting Anythingcited by 17
  15. 2024EfficientViM: Efficient Vision Mamba with Hidden State Mixer based State Space Dualitycited by 17
  16. 2025SAM2-LOVE: Segment Anything Model 2 in Language-aided Audio-Visual Scenescited by 17
  17. 2024On Efficient Variants of Segment Anything Model: A Surveycited by 16
  18. 2024Training-Free Open-Ended Object Detection and Segmentation via Attention as Promptscited by 16
  19. 20243DGS-CD: 3D Gaussian Splatting-based Change Detection for Physical Object Rearrangementcited by 15
  20. 2024Scaling Robot Policy Learning via Zero-Shot Labeling with Foundation Modelscited by 14
  21. 2023Generalizable Visual Reinforcement Learning with Segment Anything Modelcited by 12
  22. 2024ASAM: Boosting Segment Anything Model with Adversarial Tuningcited by 12
  23. 2024ASPS: Augmented Segment Anything Model for Polyp Segmentationcited by 12
  24. 2024Segment Anything with Multiple Modalitiescited by 12
  25. 2025Not All Data Are Unlearned Equallycited by 12
  26. 2024De-LightSAM: Modality-Decoupled Lightweight SAM for Generalizable Medical Segmentationcited by 11
  27. 2025ArticuBot: Learning Universal Articulated Object Manipulation Policy via Large Scale Simulationcited by 11
  28. 2025Vanish into Thin Air: Cross-prompt Universal Adversarial Attacks for SAM2cited by 11
  29. 2026Efficient Universal Perception Encodercited by 10
  30. 2024PointSeg: A Training-Free Paradigm for 3D Scene Segmentation via Foundation Modelscited by 9
  31. 2024Practical Region-level Attack against Segment Anything Modelscited by 9
  32. 2024HATs: Hierarchical Adaptive Taxonomy Segmentation for Panoramic Pathology Image Analysiscited by 9
  33. 2025WeakMCN: Multi-task Collaborative Network for Weakly Supervised Referring Expression Comprehension and Segmentationcited by 9
  34. 2025COS3D: Collaborative Open-Vocabulary 3D Segmentationcited by 9
  35. 2024Compositional Oil Spill Detection Based on Object Detector and Adapted Segment Anything Model from SAR Imagescited by 8
  36. 2024Object-level Geometric Structure Preserving for Natural Image Stitchingcited by 8
  37. 2024Eye-for-an-eye: Appearance Transfer with Semantic Correspondence in Diffusion Modelscited by 8
  38. 2024NEUSIS: A Compositional Neuro-Symbolic Framework for Autonomous Perception, Reasoning, and Planning in Complex UAV Search Missionscited by 8

50 most cited of 188 All 188 (JSON)