mlsanityData to 7 Oct 2026

PADA: Example-based Prompt Learning for on-the-fly Adaptation to Unseen Domains

arXiv 2102.12206 ↗First posted 24 Feb 2021cs.CL cs.AI cs.LGcited by 61

Not on the map. Why?

Abstract (arXiv)

Natural Language Processing algorithms have made incredible progress, but they still struggle when applied to out-of-distribution examples. We address a challenging and underexplored version of this domain adaptation problem, where an algorithm is trained on several source domains, and then applied to examples from unseen domains that are unknown at training time. Particularly, no examples, labeled or unlabeled, or any other knowledge about the target domain are available to the algorithm at training time. We present PADA: An example-based autoregressive Prompt learning algorithm for on-the-fly Any-Domain Adaptation, based on the T5 language model. Given a test example, PADA first generates a unique prompt for it and then, conditioned on this prompt, labels the example with respect to the NLP prediction task. PADA is trained to generate a prompt which is a token sequence of unrestricted length, consisting of Domain Related Features (DRFs) that characterize each of the source domains. Intuitively, the generated prompt is a unique signature that maps the test example to a semantic space spanned by the source domains. In experiments with 3 tasks (text classification and sequence tagging), for a total of 14 multi-source adaptation scenarios, PADA substantially outperforms strong baselines.

Key facts. Cited by 61 papers in the map (data through 7 Oct 2026). Builds on 36 of its 83 references that are in the map. First posted on arXiv 24 Feb 2021.

Builds on

36 of 83 references.

References · 36

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  6. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  7. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  8. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  9. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  10. 2015Domain-Adversarial Training of Neural Networkscited by 3,023
  11. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  12. 2015A large annotated corpus for learning natural language inferencecited by 1,837
24 more
  1. 2019Invariant Risk Minimizationcited by 1,832
  2. 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,366
  3. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
  4. 2019How Can We Know What Language Models Know?cited by 953
  5. 2020Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inferencecited by 934
  6. 2013Domain Generalization via Invariant Feature Representationcited by 622
  7. 2020MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfercited by 443
  8. 2016Diverse Beam Search: Decoding Diverse Solutions from Neural Sequence Modelscited by 299
  9. 2019Continual learning with hypernetworkscited by 267
  10. 2020AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Promptscited by 246
  11. 2019Distributionally Robust Language Modelingcited by 142
  12. 2020Neural Unsupervised Domain Adaptation in NLP---A Surveycited by 142
  13. 2021On Calibration and Out-of-domain Generalizationcited by 134
  14. 2019Domain Robustness in Neural Machine Translationcited by 64
  15. 2015Analysing How People Orient to and Spread Rumours in Social Media by Looking at Conversational Threadscited by 62
  16. 2018Multi-Source Domain Adaptation with Mixture of Expertscited by 62
  17. 2018Multinomial Adversarial Networks for Multi-Domain Text Classificationcited by 40
  18. 2016Does Distributionally Robust Supervised Learning Give Robust Classifiers?cited by 38
  19. 2017Zero-Shot Deep Domain Adaptationcited by 35
  20. 2020PERL: Pivot-based Domain Adaptation for Pre-trained Deep Contextualized Embedding Modelscited by 33
  21. 2020Transformer Based Multi-Source Domain Adaptationcited by 23
  22. 2020MultiCQA: Zero-Shot Transfer of Self-Supervised Text Matching Models on a Massive Scalecited by 20
  23. 2016Neural Structural Correspondence Learning for Domain Adaptationcited by 19
  24. 2020Conditioned Natural Language Generation using only Unconditioned Language Model: An Explorationcited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 16; 2023: 24; 2024: 11; 2025: 4; 2026: 4.

61 papers.

Sort:

By area:

  1. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  2. 2023A Comprehensive Survey on Test-Time Adaptation under Distribution Shiftscited by 353
  3. 2023Domain Specialization as the Key to Make Large Language Models Disruptive: A Comprehensive Surveycited by 134
  4. 2023Automatic Prompt Augmentation and Selection with Chain-of-Thought from Labeled Datacited by 119
  5. 2022Personalized Prompt Learning for Explainable Recommendationcited by 79
  6. 2022Black-box Prompt Learning for Pre-trained Language Modelscited by 70
  7. 2023Faithful Explanations of Black-box NLP Models Using LLM-generated Counterfactualscited by 44
  8. 2024PRewrite: Prompt Rewriting with Reinforcement Learningcited by 43
  9. 2023SAMAug: Point Prompt Augmentation for Segment Anything Modelcited by 38
  10. 2021LightNER: A Lightweight Tuning Paradigm for Low-resource NER via Pluggable Promptingcited by 27
  11. 2023ClickPrompt: CTR Models are Strong Prompt Generators for Adapting Language Models to CTR Predictioncited by 27
  12. 2022Multi-Prompt Alignment for Multi-Source Unsupervised Domain Adaptationcited by 25
38 more
  1. 2024Decoding Time Series with LLMs: A Multi-Agent Framework for Cross-Domain Annotationcited by 21
  2. 2022Prompt-Augmented Linear Probing: Scaling beyond the Limit of Few-shot In-Context Learnerscited by 18
  3. 2022No Parameter Left Behind: How Distillation and Model Size Affect Zero-Shot Retrievalcited by 17
  4. 2022Example-based Hypernetworks for Out-of-Distribution Generalizationcited by 16
  5. 2025AdaptiVocab: Enhancing LLM Efficiency in Focused Domains through Lightweight Vocabulary Adaptationcited by 15
  6. 2023Zero-shot Transfer of Article-aware Legal Outcome Classification for European Court of Human Rights Casescited by 14
  7. 2023Zero-shot Domain-sensitive Speech Recognition with Prompt-conditioning Fine-tuningcited by 13
  8. 2025FlexiReg: Flexible Urban Region Representation Learningcited by 13
  9. 2022On the Robustness of Dialogue History Representation in Conversational Question Answering: A Comprehensive Study and a New Prompt-based Methodcited by 12
  10. 2023Efficient Federated Prompt Tuning for Black-box Large Pre-trained Modelscited by 12
  11. 2023Measuring the Robustness of NLP Models to Domain Shiftscited by 11
  12. 2023Prompt Valuation Based on Shapley Valuescited by 10
  13. 2022A Sentence is Worth 128 Pseudo Tokens: A Semantic-Aware Contrastive Learning Framework for Sentence Embeddingscited by 9
  14. 2022Learning a Better Initialization for Soft Prompts via Meta-Learningcited by 9
  15. 2022Billions of Parameters Are Worth More Than In-domain Training Data: A case study in the Legal Case Entailment Taskcited by 8
  16. 2023Robust Question Answering against Distribution Shifts with Test-Time Adaptation: An Empirical Studycited by 8
  17. 2023Plum: Prompt Learning using Metaheuristiccited by 8
  18. 2023Speaking the Language of Your Listener: Audience-Aware Adaptation via Plug-and-Play Theory of Mindcited by 7
  19. 2023Approximating Human-Like Few-shot Learning with GPT-based Compressioncited by 7
  20. 2024Learning from Contrastive Prompts: Automated Optimization and Adaptationcited by 7
  21. 2022Extensible Prompts for Language Models on Zero-shot Language Style Customizationcited by 6
  22. 2024FIPO: Free-form Instruction-oriented Prompt Optimization with Preference Dataset and Modular Fine-tuning Schemacited by 6
  23. 2022In the Eye of the Beholder: Robust Prediction with Causal User Modelingcited by 5
  24. 2022The Causal Structure of Domain Invariant Supervised Representation Learningcited by 5
  25. 2023From Visual Prompt Learning to Zero-Shot Transfer: Mapping Is All You Needcited by 4
  26. 2023MixPro: Simple yet Effective Data Augmentation for Prompt-based Learningcited by 4
  27. 2024PAP-REC: Personalized Automatic Prompt for Recommendation Language Modelcited by 4
  28. 2023A Communication Theory Perspective on Prompting Engineering Methods for Large Language Modelscited by 3
  29. 2024Domain Generalization via Causal Adjustment for Cross-Domain Sentiment Analysiscited by 3
  30. 2024Word Matters: What Influences Domain Adaptation in Summarization?cited by 3
  31. 2022Learning Dynamic Contextualised Word Embeddings via Template-based Temporal Adaptationcited by 2
  32. 2025Multi-Domain Explainability of Preferencescited by 2
  33. 2026DetPO: In-Context Learning with Multi-Modal LLMs for Few-Shot Object Detectioncited by 2
  34. 2023PEFTT: Parameter-Efficient Fine-Tuning for low-resource Tibetan pre-trained language modelscited by 1
  35. 2023Simple and Effective Input Reformulations for Translationcited by 1
  36. 2024Context-aware Visual Storytelling with Visual Prefix Tuning and Contrastive Learningcited by 1
  37. 2025Harnessing Test-time Adaptation for NLU tasks Involving Dialects of Englishcited by 1
  38. 2026DebiasRAG: A Tuning-Free Path to Fair Generation in Large Language Models through Retrieval-Augmented Generationcited by 1

50 most cited of 61 All 61 (JSON)