mlsanityData to 7 Oct 2026

AdapterHub: A Framework for Adapting Transformers

arXiv 2007.07779 ↗First posted 15 Jul 2020cs.CLcited by 441

Not on the map. Why?

Abstract (arXiv)

The current modus operandi in NLP involves downloading and fine-tuning pre-trained models consisting of millions or billions of parameters. Storing and sharing such large trained models is expensive, slow, and time-consuming, which impedes progress towards more general and versatile NLP methods that learn from and for many tasks. Adapters -- small learnt bottleneck layers inserted within each layer of a pre-trained model -- ameliorate this issue by avoiding full fine-tuning of the entire model. However, sharing and integrating adapter layers is not straightforward. We propose AdapterHub, a framework that allows dynamic "stitching-in" of pre-trained adapters for different tasks and languages. The framework, built on top of the popular HuggingFace Transformers library, enables extremely easy and quick adaptations of state-of-the-art pre-trained models (e.g., BERT, RoBERTa, XLM-R) across tasks and languages. Downloading, sharing, and training adapters is as seamless as possible using minimal changes to the training scripts and a specialized infrastructure. Our framework enables scalable and easy access to sharing of task-specific models, particularly in low-resource scenarios. AdapterHub includes all recent adapter architectures and can be found at https://AdapterHub.ml.

Key facts. Cited by 441 papers in the map (data through 7 Oct 2026). First posted on arXiv 15 Jul 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 47; 2022: 91; 2023: 123; 2024: 93; 2025: 54; 2026: 33.

441 papers.

Sort:

By area:

  1. 2022Visual Prompt Tuningcited by 1,674
  2. 2022AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognitioncited by 714
  3. 2021Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Surveycited by 457
  4. 2023LLM-Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Modelscited by 317
  5. 2021Parameter-efficient Multi-task Fine-tuning for Transformers via Shared Hypernetworkscited by 278
  6. 2021Training Neural Networks with Fixed Sparse Maskscited by 212
  7. 2022ST-Adapter: Parameter-Efficient Image-to-Video Transfer Learningcited by 192
  8. 2022Delta Tuning: A Comprehensive Study of Parameter Efficient Methods for Pre-trained Language Modelscited by 179
  9. 2023Scaling Down to Scale Up: A Guide to Parameter-Efficient Fine-Tuningcited by 171
  10. 2024A Survey on Recent Advances in LLM-Based Multi-turn Dialogue Systemscited by 170
  11. 2021UniPELT: A Unified Framework for Parameter-Efficient Language Model Tuningcited by 162
  12. 2023Active Prompting with Chain-of-Thought for Large Language Modelscited by 154
38 more
  1. 2021On the Effectiveness of Adapter-based Tuning for Pretrained Language Model Adaptationcited by 147
  2. 2023FederatedScope-LLM: A Comprehensive Package for Fine-tuning Large Language Models in Federated Learningcited by 146
  3. 2022Lifting the Curse of Multilinguality by Pre-training Modular Transformerscited by 145
  4. 2021Composable Sparse Fine-Tuning for Cross-Lingual Transfercited by 144
  5. 2022On the Effectiveness of Parameter-Efficient Fine-Tuningcited by 140
  6. 2022AdaMix: Mixture-of-Adaptations for Parameter-efficient Model Tuningcited by 137
  7. 2023Domain Specialization as the Key to Make Large Language Models Disruptive: A Comprehensive Surveycited by 134
  8. 2024Revisiting Zeroth-Order Optimization for Memory-Efficient LLM Fine-Tuning: A Benchmarkcited by 113
  9. 2023RL4F: Generating Natural Language Feedback with Reinforcement Learning for Repairing Model Outputscited by 104
  10. 2021Sustainable Modular Debiasing of Language Modelscited by 98
  11. 2022Three things everyone should know about Vision Transformerscited by 90
  12. 2021MAGMA -- Multimodal Augmentation of Generative Models through Adapter-based Finetuningcited by 85
  13. 2022Petals: Collaborative Inference and Fine-tuning of Large Modelscited by 84
  14. 2022SparseAdapter: An Easy Approach for Improving the Parameter-Efficiency of Adapterscited by 83
  15. 2023Modular Deep Learningcited by 82
  16. 2021Self-Supervised Pretraining Improves Self-Supervised Pretrainingcited by 81
  17. 2022Efficient Methods for Natural Language Processing: A Surveycited by 78
  18. 2022Revisiting Parameter-Efficient Tuning: Are We Really There Yet?cited by 76
  19. 2021xGQA: Cross-Lingual Visual Question Answeringcited by 72
  20. 2021Trankit: A Light-Weight Transformer-based Toolkit for Multilingual Natural Language Processingcited by 68
  21. 2023Structure-informed Language Models Are Protein Designerscited by 68
  22. 2021What to Pre-Train on? Efficient Intermediate Task Selectioncited by 62
  23. 2022Contrastive Adapters for Foundation Model Group Robustnesscited by 62
  24. 2022Conv-Adapter: Exploring Parameter Efficient Transfer Learning for ConvNetscited by 62
  25. 2023Diversity-Aware Meta Visual Promptingcited by 57
  26. 2021How to Adapt Your Pretrained Multilingual Model to 1600 Languagescited by 54
  27. 2023Efficient Model Personalization in Federated Learning via Client-Specific Prompt Generationcited by 54
  28. 2021Lightweight Adapter Tuning for Multilingual Speech Translationcited by 53
  29. 2023Adapters: A Unified Library for Parameter-Efficient and Modular Transfer Learningcited by 53
  30. 2021A Survey on Green Deep Learningcited by 52
  31. 2022Polyhistor: Parameter-Efficient Multi-Task Adaptation for Dense Vision Taskscited by 52
  32. 2021A Primer on Pretrained Multilingual Language Modelscited by 49
  33. 2023AutoPEFT: Automatic Configuration Search for Parameter-Efficient Fine-Tuningcited by 49
  34. 2021Structural Adapters in Pretrained Language Models for AMR-to-text Generationcited by 48
  35. 2022BERTScore is Unfair: On Social Bias in Language Model-Based Metrics for Text Generationcited by 48
  36. 2023Detecting and Mitigating Hallucinations in Multilingual Summarisationcited by 48
  37. 2023TAIL: Task-specific Adapters for Imitation Learning with Large Pretrained Modelscited by 48
  38. 2022Memory Efficient Continual Learning with Transformerscited by 43

50 most cited of 441 All 441 (JSON)