mlsanityData to 14 Oct 2026

UDApter -- Efficient Domain Adaptation Using Adapters

arXiv 2302.03194 ↗First posted 7 Feb 2023cs.CLcited by 13

Not on the map. Why?

Abstract (arXiv)

We propose two methods to make unsupervised domain adaptation (UDA) more parameter efficient using adapters, small bottleneck layers interspersed with every layer of the large-scale pre-trained language model (PLM). The first method deconstructs UDA into a two-step process: first by adding a domain adapter to learn domain-invariant information and then by adding a task adapter that uses domain-invariant information to learn task representations in the source domain. The second method jointly learns a supervised classifier while reducing the divergence measure. Compared to strong baselines, our simple methods perform well in natural language inference (MNLI) and the cross-domain sentiment classification task. We even outperform unsupervised domain adaptation methods such as DANN and DSN in sentiment classification, and we are within 0.85% F1 for natural language inference task, by fine-tuning only a fraction of the full model parameters. We release our code at https://github.com/declare-lab/domadapter

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Builds on 29 of its 41 references that are in the map. First posted on arXiv 7 Feb 2023.

Builds on

29 of 41 references.

References · 29

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  4. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  5. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  6. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  7. 2015Domain-Adversarial Training of Neural Networkscited by 3,044
  8. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,682
  9. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,947
  10. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,565
  11. 2019SciBERT: A Pretrained Language Model for Scientific Textcited by 1,414
  12. 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,245
17 more
  1. 2016Deep CORAL: Correlation Alignment for Deep Domain Adaptationcited by 1,200
  2. 2021Towards a Unified View of Parameter-Efficient Transfer Learningcited by 872
  3. 2020AdapterFusion: Non-Destructive Task Composition for Transfer Learningcited by 788
  4. 2020MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfercited by 445
  5. 2020AdapterHub: A Framework for Adapting Transformerscited by 444
  6. 2016Domain Separation Networkscited by 320
  7. 2019Simple, Scalable Adaptation for Neural Machine Translationcited by 242
  8. 2017Central Moment Discrepancy (CMD) for Domain-Invariant Representation Learningcited by 192
  9. 2016Correlation Alignment for Unsupervised Domain Adaptationcited by 147
  10. 2021On the Effectiveness of Adapter-based Tuning for Pretrained Language Model Adaptationcited by 147
  11. 2018Co-regularized Alignment for Unsupervised Domain Adaptationcited by 39
  12. 2021Robust Transfer Learning with Pretrained Language Models through Adapterscited by 35
  13. 2021UDALM: Unsupervised Domain Adaptation through Language Modelingcited by 33
  14. 2022Continual Sequence Generation with Adaptive Compositional Modulescited by 31
  15. 2020Domain Divergences: a Survey and Empirical Analysiscited by 24
  16. 2021Unsupervised Domain Adaptation with Adaptercited by 14
  17. 2022Learning Disentangled Semantic Representations for Zero-Shot Cross-Lingual Transfer in Multilingual Machine Reading Comprehensioncited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 3; 2024: 7; 2025: 2; 2026: 1.

13 papers.

Sort:

By area:

  1. 2023Domain Specialization as the Key to Make Large Language Models Disruptive: A Comprehensive Surveycited by 135
  2. 2023Parameter-Efficient Fine-Tuning of LLaMA for the Clinical Domaincited by 38
  3. 2023Large Language Models as Annotators: Enhancing Generalization of NLP Models at Minimal Costcited by 33
  4. 2024Decoding Time Series with LLMs: A Multi-Agent Framework for Cross-Domain Annotationcited by 21
  5. 2025AdaptiVocab: Enhancing LLM Efficiency in Focused Domains through Lightweight Vocabulary Adaptationcited by 15
  6. 2024RE-Adapt: Reverse Engineered Adaptation of Large Language Modelscited by 9
  7. 2025CATS: Mitigating Correlation Shift for Multivariate Time Series Classificationcited by 7
  8. 2024How Useful is Continued Pre-Training for Generative Unsupervised Domain Adaptation?cited by 5
  9. 2024DAdEE: Unsupervised Domain Adaptation in Early Exit PLMscited by 5
  10. 2024What the Weight?! A Unified Framework for Zero-Shot Knowledge Compositioncited by 4
  11. 2024Multi-Task Domain Adaptation for Language Grounding with 3D Objectscited by 2
  12. 2024Fixed and Adaptive Simultaneous Machine Translation Strategies Using Adapterscited by 2
1 more
  1. 2026MixedPEFT: Combining Multiple PEFT Methods with Mixed Objectives for Unsupervised Domain Adaptationcited by 0