mlsanityData to 14 Oct 2026

Adapters Strike Back

arXiv 2406.06820 ↗First posted 10 Jun 2024cs.CV cs.LGcited by 16

Not on the map. Why?

Abstract (arXiv)

Adapters provide an efficient and lightweight mechanism for adapting trained transformer models to a variety of different tasks. However, they have often been found to be outperformed by other adaptation mechanisms, including low-rank adaptation. In this paper, we provide an in-depth study of adapters, their internal structure, as well as various implementation choices. We uncover pitfalls for using adapters and suggest a concrete, improved adapter architecture, called Adapter+, that not only outperforms previous adapter implementations but surpasses a number of other, more complex adaptation mechanisms in several challenging settings. Despite this, our suggested adapter is highly robust and, unlike previous work, requires little to no manual intervention when addressing a novel scenario. Adapter+ reaches state-of-the-art average accuracy on the VTAB benchmark, even without a per-task hyperparameter optimization.

Key facts. Cited by 16 papers in the map (data through 14 Oct 2026). Builds on 46 of its 68 references that are in the map. First posted on arXiv 10 Jun 2024.

Builds on

46 of 68 references.

References · 46

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  4. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  6. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  7. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  8. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  9. 2020End-to-End Object Detection with Transformerscited by 7,414
  10. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  11. 2016Layer Normalizationcited by 5,669
  12. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,475
34 more
  1. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
  2. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
  3. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  4. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  5. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  6. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  7. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,611
  8. 2016Gaussian Error Linear Units (GELUs)cited by 3,502
  9. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,940
  10. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,269
  11. 2013Describing Textures in the Wildcited by 2,125
  12. 2021Vision Transformers for Dense Predictioncited by 1,803
  13. 2022Visual Prompt Tuningcited by 1,681
  14. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,496
  15. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,457
  16. 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,245
  17. 2016Deep Networks with Stochastic Depthcited by 979
  18. 2021Towards a Unified View of Parameter-Efficient Transfer Learningcited by 872
  19. 2017Remote Sensing Image Scene Classification: Benchmark and State of the Artcited by 797
  20. 2020AdapterFusion: Non-Destructive Task Composition for Transfer Learningcited by 788
  21. 2022AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognitioncited by 719
  22. 2023Scaling Vision Transformers to 22 Billion Parameterscited by 694
  23. 2021How to train your ViT? Data, Augmentation, and Regularization in Vision Transformerscited by 569
  24. 2017Learning multiple visual domains with residual adapterscited by 557
  25. 2019A Large-scale Study of Representation Learning with the Visual Task Adaptation Benchmarkcited by 425
  26. 2013DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognitioncited by 424
  27. 2018Rotation Equivariant CNNs for Digital Pathologycited by 422
  28. 2022Scaling & Shifting Your Features: A New Baseline for Efficient Model Tuningcited by 294
  29. 2022Neural Prompt Searchcited by 135
  30. 2022FacT: Factor-Tuning for Lightweight Adaptation on Vision Transformercited by 133
  31. 2017Fine-Grained Car Detection for Visual Census Estimationcited by 76
  32. 2019Depthwise Convolution is All You Need for Learning Multiple Visual Domainscited by 36
  33. 2023Consolidator: Mergeable Adapter with Grouped Connections for Visual Adaptationcited by 18
  34. 2019NetTailor: Tuning the Architecture, Not Just the Weightscited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 10; 2026: 3.

16 papers.

Sort:

By area:

  1. 2024ExPLoRA: Parameter-Efficient Extended Pre-Training to Adapt Vision Transformers under Domain Shiftscited by 16
  2. 2024MambaPEFT: Exploring Parameter-Efficient Fine-Tuning for Mambacited by 11
  3. 2025PRO-VPT: Distribution-Adaptive Visual Prompt Tuning via Prompt Relocationcited by 9
  4. 2025Efficient Self-Supervised Learning for Earth Observation via Dynamic Dataset Curationcited by 7
  5. 2025Referring Expression Comprehension for Small Objectscited by 6
  6. 2025On Geometry-Enhanced Parameter-Efficient Fine-Tuning for 3D Scene Segmentationcited by 4
  7. 2025Efficient Adaptation of Pre-trained Vision Transformer underpinned by Approximately Orthogonal Fine-Tuning Strategycited by 3
  8. 2025Fantastic Features and Where to Find Them: A Probing Method to combine Features from Multiple Foundation Modelscited by 3
  9. 2024Multiple-Exit Tuning: Towards Inference-Efficient Adaptation for Vision Transformercited by 2
  10. 2025State Space Prompting via Gathering and Spreading Spatio-Temporal Information for Video Understandingcited by 2
  11. 2025APLA: A Simple Adaptation Method for Vision Transformerscited by 1
  12. 2025COLORA: Efficient Fine-Tuning for Convolutional Models with a Study Case on Optical Coherence Tomography Image Classificationcited by 1
4 more
  1. 2025Collaborative Low-Rank Adaptation for Pre-Trained Vision Transformerscited by 0
  2. 2026CUE: Concept-Aware Multi-Label Expansion to Mitigate Concept Confusion in Long-Tailed Learningcited by 0
  3. 2026RootQuantV2: Adapting a Vision Foundation Model for Root-Trait Regression from Minirhizotron Imagerycited by 0
  4. 2026Loop Dropout: Regularizing Shared Updates in Looped Language Modelscited by 0