mlsanityData to 7 Oct 2026

Conv-Adapter: Exploring Parameter Efficient Transfer Learning for ConvNets

arXiv 2208.07463 ↗First posted 15 Aug 2022cs.CV cs.AIcited by 62

Not on the map. Why?

Abstract (arXiv)

While parameter efficient tuning (PET) methods have shown great potential with transformer architecture on Natural Language Processing (NLP) tasks, their effectiveness with large-scale ConvNets is still under-studied on Computer Vision (CV) tasks. This paper proposes Conv-Adapter, a PET module designed for ConvNets. Conv-Adapter is light-weight, domain-transferable, and architecture-agnostic with generalized performance on different tasks. When transferring on downstream tasks, Conv-Adapter learns tasks-specific feature modulation to the intermediate representations of backbones while keeping the pre-trained parameters frozen. By introducing only a tiny amount of learnable parameters, e.g., only 3.5% full fine-tuning parameters of ResNet50. It can also be applied for transformer-based backbones. Conv-Adapter outperforms previous PET baseline methods and achieves comparable or surpasses the performance of full fine-tuning on 23 classification tasks of various domains. It also presents superior performance on the few-shot classification with an average margin of 3.39%. Beyond classification, Conv-Adapter can generalize to detection and segmentation tasks with more than 50% reduction of parameters but comparable performance to the traditional full fine-tuning.

Key facts. Cited by 62 papers in the map (data through 7 Oct 2026). Builds on 47 of its 67 references that are in the map. First posted on arXiv 15 Aug 2022.

Builds on

47 of 67 references.

References · 47

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  5. 2020Language Models are Few-Shot Learnerscited by 32,199
  6. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  7. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  8. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  9. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  10. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  11. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  12. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
35 more
  1. 2014Going Deeper with Convolutionscited by 4,231
  2. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  3. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  4. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  5. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  6. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  7. 2022A ConvNet for the 2020scited by 3,545
  8. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  9. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  10. 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,154
  11. 2013Fine-Grained Visual Classification of Aircraftcited by 1,701
  12. 2019Similarity of Neural Network Representations Revisitedcited by 1,695
  13. 2022Visual Prompt Tuningcited by 1,674
  14. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,610
  15. 2019MMDetection: Open MMLab Detection Toolbox and Benchmarkcited by 1,379
  16. 2018Unified Perceptual Parsing for Scene Understandingcited by 1,265
  17. 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,234
  18. 2021Swin Transformer V2: Scaling Up Capacity and Resolutioncited by 1,215
  19. 2021EfficientNetV2: Smaller Models and Faster Trainingcited by 1,127
  20. 2020Designing Network Design Spacescited by 954
  21. 2021Towards a Unified View of Parameter-Efficient Transfer Learningcited by 863
  22. 2020AdapterFusion: Non-Destructive Task Composition for Transfer Learningcited by 780
  23. 2019Big Transfer (BiT): General Visual Representation Learningcited by 766
  24. 2022AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognitioncited by 714
  25. 2021Do Vision Transformers See Like Convolutional Neural Networks?cited by 671
  26. 2018ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardwarecited by 666
  27. 2020ResNeSt: Split-Attention Networkscited by 445
  28. 2020AdapterHub: A Framework for Adapting Transformerscited by 441
  29. 2019A Large-scale Study of Representation Learning with the Visual Task Adaptation Benchmarkcited by 419
  30. 2022Exploring Visual Prompts for Adapting Large-Scale Modelscited by 271
  31. 2018Efficient parametrization of multi-domain deep neural networkscited by 168
  32. 2018Adversarial Reprogramming of Neural Networkscited by 115
  33. 2021Differentiable Prompt Makes Pre-trained Language Models Better Few-shot Learnerscited by 115
  34. 2019Depthwise Convolution is All You Need for Learning Multiple Visual Domainscited by 36
  35. 2018K for the Price of 1: Parameter-efficient Multi-task and Transfer Learningcited by 34

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 17; 2024: 18; 2025: 11; 2026: 13.

62 papers.

Sort:

By area:

  1. 2022LPT: Long-tailed Prompt Tuning for Image Classificationcited by 69
  2. 2023End-to-End Temporal Action Detection with 1B Parameters Across 1000 Framescited by 47
  3. 2024KVQ: Kwai Video Quality Assessment for Short-form Videoscited by 38
  4. 2022Cross-Modal Adapter for Vision-Language Retrievalcited by 35
  5. 2023Visual Tuningcited by 34
  6. 2023Parameter-efficient is not sufficient: Exploring Parameter, Memory, and Time Efficient Adapter Tuning for Dense Predictionscited by 24
  7. 2023Parameter-Efficient Transfer Learning for Remote Sensing Image-Text Retrievalcited by 24
  8. 2023VL-PET: Vision-and-Language Parameter-Efficient Tuning via Granularity Controlcited by 21
  9. 2024Natural Language Can Help Bridge the Sim2Real Gapcited by 21
  10. 2023SCT: A Simple Baseline for Parameter-Efficient Fine-Tuning via Salient Channelscited by 20
  11. 2023Rethinking Vision Transformer and Masked Autoencoder in Multimodal Face Anti-Spoofingcited by 19
  12. 2023PECoP: Parameter Efficient Continual Pretraining for Action Quality Assessmentcited by 19
38 more
  1. 2023Less is More: Selective Layer Finetuning with SubTuningcited by 13
  2. 2023When Geoscience Meets Foundation Models: Towards General Geoscience Artificial Intelligence Systemcited by 13
  3. 2024Bayesian-guided Label Mapping for Visual Reprogrammingcited by 12
  4. 2022Exploring Parameter-Efficient Fine-Tuning to Enable Foundation Models in Federated Learningcited by 11
  5. 2023Adaptive FSS: A Novel Few-Shot Segmentation Framework via Prototype Enhancementcited by 10
  6. 2024DA-Ada: Learning Domain-Aware Adapter for Domain Adaptive Object Detectioncited by 8
  7. 2023Prompt-ICM: A Unified Framework towards Image Coding for Machines with Task-driven Promptscited by 7
  8. 2024Exploring Semantic Consistency and Style Diversity for Domain Generalized Semantic Segmentationcited by 7
  9. 2023DR-Tune: Improving Fine-tuning of Pretrained Visual Models by Distribution Regularization with Semantic Calibrationcited by 6
  10. 2023Revisit Parameter-Efficient Transfer Learning: A Two-Stage Paradigmcited by 5
  11. 2023TaCA: Upgrading Your Visual Foundation Model with Task-agnostic Compatible Adaptercited by 5
  12. 2024Efficient Fine-tuning of Audio Spectrogram Transformers via Soft Mixture of Adapterscited by 5
  13. 2025Analytical Survey of Learning with Low-Resource Data: From Analysis to Investigationcited by 5
  14. 2023A Foundation Model for General Moving Object Segmentation in Medical Imagescited by 4
  15. 2024ALoRE: Efficient Visual Adaptation via Aggregating Low Rank Expertscited by 4
  16. 2024ConSept: Continual Semantic Segmentation via Adapter-based Vision Transformercited by 3
  17. 2024Editable-DeepSC: Reliable Cross-Modal Semantic Communications for Facial Editingcited by 3
  18. 2025Generalizable Multispectral Land Cover Classification via Frequency-Aware Mixture of Low-Rank Token Expertscited by 3
  19. 2025SafeTriage: Facial Video De-identification for Privacy-Preserving Stroke Triagecited by 3
  20. 2024Rethinking Attention-Based Multiple Instance Learning for Whole-Slide Pathological Image Classification: An Instance Attribute Viewpointcited by 2
  21. 2025VLOD-TTA: Test-Time Adaptation of Vision-Language Object Detectorscited by 2
  22. 2025Color3D: Controllable and Consistent 3D Colorization with Personalized Colorizercited by 2
  23. 2023Exploring Visual Prompts for Whole Slide Image Classification with Multiple Instance Learningcited by 1
  24. 2024Hierarchical Prompts for Rehearsal-free Continual Learningcited by 1
  25. 2024Pre-training Everywhere: Parameter-Efficient Fine-Tuning for Medical Image Analysis via Target Parameter Pre-trainingcited by 1
  26. 2024LoKO: Low-Rank Kalman Optimizer for Online Fine-Tuning of Large Modelscited by 1
  27. 2024Enhancing Parameter-Efficient Fine-Tuning of Vision Transformers through Frequency-Based Adaptationcited by 1
  28. 2024Parameter-efficient Fine-tuning for improved Convolutional Baseline for Brain Tumor Segmentation in Sub-Saharan Africa Adult Glioma Datasetcited by 1
  29. 2025An Online Adaptation Method for Robust Depth Estimation and Visual Odometry in the Open Worldcited by 1
  30. 2025MedSeqFT: Sequential Fine-tuning Foundation Models for 3D Medical Image Segmentationcited by 1
  31. 2025Silhouette-based Gait Foundation Modelcited by 1
  32. 2026S2FT: Parameter-Efficient Fine-Tuning in Sparse Spectrum Domaincited by 1
  33. 2024Mobius: A High Efficient Spatial-Temporal Parallel Training Paradigm for Text-to-Video Generation Taskcited by 0
  34. 2024MergeRepair: An Exploratory Study on Merging Task-Specific Adapters in Code LLMs for Automated Program Repaircited by 0
  35. 2024PROFIT: A Specialized Optimizer for Deep Fine Tuningcited by 0
  36. 2025Fully Exploiting Vision Foundation Model's Profound Prior Knowledge for Generalizable RGB-Depth Driving Scene Parsingcited by 0
  37. 2025S2A: A Unified Framework for Parameter and Memory Efficient Transfer Learningcited by 0
  38. 2025A Tale of Two Experts: Cooperative Learning for Source-Free Unsupervised Domain Adaptationcited by 0

50 most cited of 62 All 62 (JSON)