mlsanityData to 7 Oct 2026

Adapters: A Unified Library for Parameter-Efficient and Modular Transfer Learning

arXiv 2311.11077 ↗First posted 18 Nov 2023cs.CL cs.AI cs.LGcited by 53

Not on the map. Why?

Abstract (arXiv)

We introduce Adapters, an open-source library that unifies parameter-efficient and modular transfer learning in large language models. By integrating 10 diverse adapter methods into a unified interface, Adapters offers ease of use and flexible configuration. Our library allows researchers and practitioners to leverage adapter modularity through composition blocks, enabling the design of complex adapter setups. We demonstrate the library's efficacy by evaluating its performance against full fine-tuning on various NLP tasks. Adapters provides a powerful tool for addressing the challenges of conventional fine-tuning paradigms and promoting more efficient and modular transfer learning. The library is available via https://adapterhub.ml/adapters.

Key facts. Cited by 53 papers in the map (data through 7 Oct 2026). Builds on 35 of its 47 references that are in the map. First posted on arXiv 18 Nov 2023.

Builds on

35 of 47 references.

References · 35

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  6. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  7. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  8. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  9. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  10. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  11. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  12. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
23 more
  1. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
  2. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,538
  3. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,308
  4. 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,234
  5. 2022Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningcited by 886
  6. 2021Towards a Unified View of Parameter-Efficient Transfer Learningcited by 863
  7. 2020AdapterFusion: Non-Destructive Task Composition for Transfer Learningcited by 780
  8. 2017Learning multiple visual domains with residual adapterscited by 553
  9. 2021Compacter: Efficient Low-Rank Hypercomplex Adapter Layerscited by 458
  10. 2020MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfercited by 443
  11. 2020AdapterHub: A Framework for Adapting Transformerscited by 441
  12. 2019Cosmos QA: Machine Reading Comprehension with Contextual Commonsense Reasoningcited by 281
  13. 2020AdapterDrop: On the Efficiency of Adapters in Transformerscited by 223
  14. 2023Scaling Down to Scale Up: A Guide to Parameter-Efficient Fine-Tuningcited by 171
  15. 2021UniPELT: A Unified Framework for Parameter-Efficient Language Model Tuningcited by 162
  16. 2022Lifting the Curse of Multilinguality by Pre-training Modular Transformerscited by 145
  17. 2022AdaMix: Mixture-of-Adaptations for Parameter-efficient Model Tuningcited by 137
  18. 2020UNKs Everywhere: Adapting Multilingual Language Models to New Scriptscited by 113
  19. 2023Conditional Adapters: Parameter-efficient Transfer Learning with Fast Inferencecited by 69
  20. 2021What to Pre-Train on? Efficient Intermediate Task Selectioncited by 62
  21. 2021Efficient Test Time Adapter Ensembling for Low-resource Language Varietiescited by 26
  22. 2021Single-dataset Experts for Multi-dataset Question Answeringcited by 15
  23. 2023PEFT-Ref: A Modular Reference Architecture and Typology for Parameter-Efficient Finetuning Techniquescited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 23; 2025: 22; 2026: 8.

53 papers.

Sort:

By area:

  1. 2024Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Surveycited by 641
  2. 2024Towards Modular LLMs by Building and Reusing a Library of LoRAscited by 72
  3. 2024Recent Advances of Foundation Language Models-based Continual Learning: A Surveycited by 46
  4. 2024Segment Any Text: A Universal Approach for Robust, Efficient and Adaptable Sentence Segmentationcited by 41
  5. 2024Zero-Shot Tokenizer Transfercited by 30
  6. 2024FeDeRA:Efficient Fine-tuning of Language Models in Federated Learning Leveraging Weight Decompositioncited by 26
  7. 2025Efficient Model Editing with Task-Localized Sparse Fine-tuningcited by 16
  8. 2025OccVLA: Vision-Language-Action Model with Implicit 3D Occupancy Supervisioncited by 16
  9. 2024Automated Text Scoring in the Age of Generative AI for the GPU-poorcited by 10
  10. 2025Capturing Individual Human Preferences with Reward Featurescited by 8
  11. 2024HyperLoader: Integrating Hypernetwork-Based LoRA and Adapter Layers into Multi-Task Transformers for Sequence Labellingcited by 6
  12. 2025Less but Better: Parameter-Efficient Fine-Tuning of Large Language Models for Personality Detectioncited by 6
38 more
  1. 2025When Dynamic Data Selection Meets Data Augmentationcited by 6
  2. 2024AQuA -- Combining Experts' and Non-Experts' Views To Assess Deliberation Quality in Online Discussions Using LLMscited by 5
  3. 2024M2QA: Multi-domain Multilingual Question Answeringcited by 5
  4. 2024Transfer Learning for Finetuning Large Language Modelscited by 5
  5. 2025Improving LoRA with Variational Learningcited by 5
  6. 2026KOCO-BENCH: Can Large Language Models Leverage Domain Knowledge in Software Development?cited by 5
  7. 2024Unlocking Parameter-Efficient Fine-Tuning for Low-Resource Language Translationcited by 4
  8. 2024Glider: Global and Local Instruction-Driven Expert Routercited by 4
  9. 2025ObscuraCoder: Powering Efficient Code LM Pre-Training Via Obfuscation Groundingcited by 4
  10. 2025On Geometry-Enhanced Parameter-Efficient Fine-Tuning for 3D Scene Segmentationcited by 4
  11. 2024Towards Zero-Shot Multimodal Machine Translationcited by 3
  12. 2024CogSteer: Cognition-Inspired Selective Layer Intervention for Efficiently Steering Large Language Modelscited by 3
  13. 2024Modular Sentence Encoders: Separating Language Specialization from Cross-Lingual Alignmentcited by 2
  14. 2024Train More Parameters But Mind Their Placement: Insights into Language Adaptation with PEFTcited by 2
  15. 2025Topological derivative approach for deep neural network architecture adaptationcited by 2
  16. 2025K-DeCore: Facilitating Knowledge Transfer in Continual Structured Knowledge Reasoning via Knowledge Decouplingcited by 2
  17. 2025Facilitating Cognitive Accessibility with LLMs: A Multi-Task Approach to Easy-to-Read Text Generationcited by 2
  18. 2024GNNavi: Navigating the Information Flow in Large Language Models by Graph Neural Networkcited by 1
  19. 2025Outlook Towards Deployable Continual Learning for Particle Acceleratorscited by 1
  20. 2025Orthogonal Projection Subspace to Aggregate Online Prior-knowledge for Continual Test-time Adaptationcited by 1
  21. 2025Bayesian Low-Rank Factorization for Robust Model Adaptationcited by 1
  22. 2026Typologically Informed Parameter Aggregationcited by 1
  23. 2026Reconstructing Training Data from Adapter-based Federated Large Language Modelscited by 1
  24. 2026Few Shots Text to Image Retrieval: New Benchmarking Dataset and Optimization Methodscited by 1
  25. 2024Unplug and Play Language Models: Decomposing Experts in Language Models at Inference Timecited by 0
  26. 2024TartuNLP @ SIGTYP 2024 Shared Task: Adapting XLM-RoBERTa for Ancient and Historical Languagescited by 0
  27. 2024Enhancing Robustness of Vision-Language Models through Orthogonality Learning and Self-Regularizationcited by 0
  28. 2024MergeRepair: An Exploratory Study on Merging Task-Specific Adapters in Code LLMs for Automated Program Repaircited by 0
  29. 2024PETapter: Leveraging PET-style classification heads for modular few-shot parameter-efficient fine-tuningcited by 0
  30. 2025How to Tune a Multilingual Encoder Model for Germanic Languages: A Study of PEFT, Full Fine-Tuning, and Language Adapterscited by 0
  31. 2025Plausibility Vaccine: Injecting LLM Knowledge for Event Plausibilitycited by 0
  32. 2025PartStickers: Generating Parts of Objects for Rapid Prototypingcited by 0
  33. 2025Federated Foundation Language Model Post-Training Should Focus on Open-Source Modelscited by 0
  34. 2025RelayFormer: A Unified Local-Global Attention Framework for Scalable Image and Video Manipulation Localizationcited by 0
  35. 2025A Systematic Review on the Generative AI Applications in Human Medical Genomicscited by 0
  36. 2025PEFT-Factory: Unified Parameter-Efficient Fine-Tuning of Autoregressive Large Language Modelscited by 0
  37. 2025GRASP: GRouped Activation Shared Parameterization for Parameter-Efficient Fine-Tuning and Robust Inference of Transformerscited by 0
  38. 2026FairNVT: Fair Classification via Noise Injection in Vision Transformerscited by 0

50 most cited of 53 All 53 (JSON)