mlsanityData to 7 Oct 2026

Convolution Meets LoRA: Parameter Efficient Finetuning for Segment Anything Model

arXiv 2401.17868 ↗First posted 31 Jan 2024cs.CV cs.LGcited by 67

Not on the map. Why?

Abstract (arXiv)

The Segment Anything Model (SAM) stands as a foundational framework for image segmentation. While it exhibits remarkable zero-shot generalization in typical scenarios, its advantage diminishes when applied to specialized domains like medical imagery and remote sensing. To address this limitation, this paper introduces Conv-LoRA, a simple yet effective parameter-efficient fine-tuning approach. By integrating ultra-lightweight convolutional parameters into Low-Rank Adaptation (LoRA), Conv-LoRA can inject image-related inductive biases into the plain ViT encoder, further reinforcing SAM's local prior assumption. Notably, Conv-LoRA not only preserves SAM's extensive segmentation knowledge but also revives its capacity of learning high-level image semantics, which is constrained by SAM's foreground-background segmentation pretraining. Comprehensive experimentation across diverse benchmarks spanning multiple domains underscores Conv-LoRA's superiority in adapting SAM to real-world semantic segmentation tasks.

Key facts. Cited by 67 papers in the map (data through 7 Oct 2026). Builds on 64 of its 75 references that are in the map. First posted on arXiv 31 Jan 2024.

Builds on

64 of 75 references.

References · 64

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  6. 2023Segment Anythingcited by 8,158
  7. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  8. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  9. 2014Going Deeper with Convolutionscited by 4,231
  10. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  11. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  12. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
38 more
  1. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  2. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,239
  3. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  4. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,923
  5. 2016Pyramid Scene Parsing Networkcited by 2,472
  6. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,338
  7. 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,154
  8. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,960
  9. 2022Visual Prompt Tuningcited by 1,674
  10. 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,234
  11. 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,148
  12. 2018Dual Attention Network for Scene Segmentationcited by 1,001
  13. 2022Token Merging: Your ViT But Fastercited by 995
  14. 2021CvT: Introducing Convolutions to Vision Transformerscited by 946
  15. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 890
  16. 2021PVT v2: Improved Baselines with Pyramid Vision Transformercited by 749
  17. 2021Scaling Vision with Sparse Mixture of Expertscited by 740
  18. 2023Scaling Vision Transformers to 22 Billion Parameterscited by 690
  19. 2021Do Vision Transformers See Like Convolutional Neural Networks?cited by 671
  20. 2022Mixture-of-Experts with Expert Choice Routingcited by 611
  21. 2019Kvasir-SEG: A Segmented Polyp Datasetcited by 588
  22. 2022Vision Transformer Adapter for Dense Predictionscited by 564
  23. 2017Skin Lesion Analysis Toward Melanoma Detection: A Challenge at the 2017 International Symposium on Biomedical Imaging (ISBI), Hosted by the International Skin Imaging Collaboration (ISIC)cited by 549
  24. 2023Medical SAM Adapter: Adapting Segment Anything Model for Medical Image Segmentationcited by 485
  25. 2021LeViT: a Vision Transformer in ConvNet's Clothing for Faster Inferencecited by 475
  26. 2020PraNet: Parallel Reverse Attention Network for Polyp Segmentationcited by 460
  27. 2021VLMo: Unified Vision-Language Pre-Training with Mixture-of-Modality-Expertscited by 424
  28. 2020Parameter-Efficient Transfer Learning with Diff Pruningcited by 351
  29. 2022How Do Vision Transformers Work?cited by 305
  30. 2023Side Adapter Network for Open-Vocabulary Semantic Segmentationcited by 296
  31. 2022Scaling & Shifting Your Features: A New Baseline for Efficient Model Tuningcited by 292
  32. 2023Customized Segment Anything Model for Medical Image Segmentationcited by 277
  33. 2021TransFuse: Fusing Transformers and CNNs for Medical Image Segmentationcited by 258
  34. 2022LST: Ladder Side-Tuning for Parameter and Memory Efficient Transfer Learningcited by 228
  35. 2021Anabranch Network for Camouflaged Object Segmentationcited by 227
  36. 2021Co-Scale Conv-Attentional Image Transformerscited by 224
  37. 2016A Benchmark for Endoluminal Scene Segmentation of Colonoscopy Imagescited by 220
  38. 2021Concealed Object Detectioncited by 212

50 most cited of 64 All 64 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 25; 2025: 25; 2026: 17.

67 papers.

Sort:

By area:

  1. 2025DICEPTION: A Generalist Diffusion Model for Visual Perceptual Taskscited by 37
  2. 2024AutoGluon-Multimodal (AutoMM): Supercharging Multimodal AutoML with Foundation Modelscited by 29
  3. 2024MoE-FFD: Mixture of Experts for Generalized and Parameter-Efficient Face Forgery Detectioncited by 27
  4. 2024UN-SAM: Universal Prompt-Free Segmentation for Generalized Nuclei Imagescited by 22
  5. 2025MLZero: A Multi-Agent System for End-to-end Machine Learning Automationcited by 22
  6. 2025AuroRA: Breaking Low-Rank Bottleneck of LoRA with Nonlinear Mappingcited by 22
  7. 2025LiMoE: Mixture of LiDAR Representation Learners from Automotive Scenescited by 19
  8. 2024GEAL: Generalizable 3D Affordance Learning with Cross-Modal Consistencycited by 17
  9. 2025ROS-SAM: High-Quality Interactive Segmentation for Remote Sensing Moving Objectcited by 15
  10. 2024UniRestorer: Universal Image Restoration via Adaptively Estimating Image Degradation at Proper Granularitycited by 12
  11. 2024Beyond Task Vectors: Selective Task Arithmetic Based on Importance Metricscited by 11
  12. 2024Low-Rank Interconnected Adaptation across Layerscited by 10
38 more
  1. 2024Agent Skill Acquisition for Large Language Models via CycleQDcited by 10
  2. 2024FlexEvent: Towards Flexible Event-Frame Object Detection at Varying Operational Frequenciescited by 10
  3. 2024PhysMLE: Generalizable and Priors-Inclusive Multi-task Remote Physiological Measurementcited by 9
  4. 2024Modality-Specialized Synergizers for Interleaved Vision-Language Generalistscited by 9
  5. 2024EvRT-DETR: Latent Space Adaptation of Image Detectors for Event-based Visioncited by 9
  6. 2025LM-Searcher: Cross-domain Neural Architecture Search with LLMs via Unified Numerical Encodingcited by 7
  7. 2024Task-Aware Low-Rank Adaptation of Segment Anything Modelcited by 6
  8. 2024Generalized SAM: Efficient Fine-Tuning of SAM for Variable Input Image Sizescited by 6
  9. 2024Sam2Rad: A Segmentation Model for Medical Images with Learnable Promptscited by 6
  10. 2025Talk2Event: Grounded Understanding of Dynamic Scenes from Event Camerascited by 6
  11. 2025MoE3D: Mixture of Experts meets Multi-Modal 3D Understandingcited by 6
  12. 2024NuSegDG: Integration of Heterogeneous Space and Gaussian Kernel for Domain-Generalized Nuclei Segmentationcited by 5
  13. 2025TADFormer : Task-Adaptive Dynamic Transformer for Efficient Multi-Task Learningcited by 5
  14. 2025PHLoRA: data-free Post-hoc Low-Rank Adapter extraction from full-rank checkpointcited by 5
  15. 2025UniUltra: Interactive Parameter-Efficient SAM2 for Universal Ultrasound Segmentationcited by 5
  16. 2026Bilevel Layer-Positioning LoRA for Real Image Dehazingcited by 5
  17. 2024VIRL: Volume-Informed Representation Learning towards Few-shot Manufacturability Estimationcited by 4
  18. 2024DEeR: Deviation Eliminating and Noise Regulating for Privacy-preserving Federated Low-rank Adaptationcited by 4
  19. 2024Adapting Segment Anything Model (SAM) to Experimental Datasets via Fine-Tuning on GAN-based Simulation: A Case Study in Additive Manufacturingcited by 4
  20. 2025RS2-SAM2: Customized SAM2 for Referring Remote Sensing Image Segmentationcited by 4
  21. 2024Adaptive Blind All-in-One Image Restorationcited by 3
  22. 2025SAM Guided Semantic and Motion Changed Region Mining for Remote Sensing Change Captioningcited by 3
  23. 20251LoRA: Summation Compression for Very Low-Rank Adaptationcited by 2
  24. 2025Clear Nights Ahead: Towards Multi-Weather Nighttime Image Restorationcited by 2
  25. 2025PanMatch: Unleashing the Potential of Large Vision Models for Unified Matching Modelscited by 2
  26. 2026SegMoTE: Token-Level Mixture of Experts for Medical Image Segmentationcited by 2
  27. 2024Pre-training Everywhere: Parameter-Efficient Fine-Tuning for Medical Image Analysis via Target Parameter Pre-trainingcited by 1
  28. 2024A Survey of Low-shot Vision-Language Model Adaptation via Representer Theoremcited by 1
  29. 2024EP-SAM: Weakly Supervised Histopathology Segmentation via Enhanced Prompt with Segment Anythingcited by 1
  30. 2024PETAH: Parameter Efficient Task Adaptation for Hybrid Transformers in a resource-limited Contextcited by 1
  31. 2025InfoSAM: Fine-Tuning the Segment Anything Model from An Information-Theoretic Perspectivecited by 1
  32. 2025Landmark Detection for Medical Images using a General-purpose Segmentation Modelcited by 1
  33. 2025SAM-PTx: Text-Guided Fine-Tuning of SAM with Parameter-Efficient, Parallel-Text Adapterscited by 1
  34. 2026Frequency Switching Mechanism for Parameter-E!cient Multi-Task Learningcited by 1
  35. 2026Live Interactive Training for Video Segmentationcited by 1
  36. 2026Prompt-Free and Efficient SAM2 Adaptation for Biomedical Semantic Segmentation via Dual Adapterscited by 1
  37. 2026XCT-SAM: Sequential Parameter-Efficient Domain Adaptation of SAM for Industrial XCT Defect Segmentationcited by 1
  38. 2026CROSS: Cascaded Distillation and Dual-Constraint Grounding for Remote Sensing Referring Segmentationcited by 1

50 most cited of 67 All 67 (JSON)