mlsanityData to 14 Oct 2026

Adapt in Contexts: Retrieval-Augmented Domain Adaptation via In-Context Learning

arXiv 2311.11551 ↗First posted 20 Nov 2023cs.CLcited by 16

On the map: Retrieval-augmented generation → Generation and context integration

Abstract (arXiv)

Large language models (LLMs) have showcased their capability with few-shot inference known as in-context learning. However, in-domain demonstrations are not always readily available in real scenarios, leading to cross-domain in-context learning. Besides, LLMs are still facing challenges in long-tail knowledge in unseen and unfamiliar domains. The above limitations demonstrate the necessity of Unsupervised Domain Adaptation (UDA). In this paper, we study the UDA problem under an in-context learning setting to adapt language models from the source domain to the target domain without any target labels. The core idea is to retrieve a subset of cross-domain elements that are the most similar to the query, and elicit language model to adapt in an in-context manner by learning both target domain distribution and the discriminative task signal simultaneously with the augmented cross-domain in-context examples. We devise different prompting and training strategies, accounting for different LM architectures to learn the target distribution via language modeling. With extensive experiments on Sentiment Analysis (SA) and Named Entity Recognition (NER) tasks, we thoroughly study the effectiveness of ICL for domain transfer and demonstrate significant improvements over baseline models.

Key facts. Cited by 16 papers in the map (data through 14 Oct 2026). Placed in Generation and context integration, one of 15 directions in Retrieval-augmented generation. Builds on 31 of its 46 references that are in the map. First posted on arXiv 20 Nov 2023.

Builds on

31 of 46 references.

Closest ancestors · 4

Same direction, newest first

  1. 2023REPLUG: Retrieval-Augmented Black-Box Language Models30 Jan
  2. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Models5 Aug
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks22 May
  4. 2020REALM: Retrieval-Augmented Language Model Pre-Training10 Feb

Other references · 27

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  6. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  7. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  8. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  9. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  10. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  11. 2015Domain-Adversarial Training of Neural Networkscited by 3,044
  12. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
15 more
  1. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,173
  2. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,947
  3. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,434
  4. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,392
  5. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,195
  6. 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 938
  7. 2015Image-based Recommendations on Styles and Substitutescited by 706
  8. 2021Learning To Retrieve Prompts for In-Context Learningcited by 648
  9. 2016Neural Architectures for Named Entity Recognitioncited by 550
  10. 2016End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRFcited by 341
  11. 2021Improving Named Entity Recognition by External Context Retrieving and Cooperative Learningcited by 76
  12. 2019An Embarrassingly Simple Approach for Transfer Learning from Pretrained Language Modelscited by 39
  13. 2021UDALM: Unsupervised Domain Adaptation through Language Modelingcited by 33
  14. 2018Adaptive Semi-supervised Learning for Cross-domain Sentiment Classificationcited by 16
  15. 2020Effective Unsupervised Domain Adaptation with Adversarially Trained Language Modelscited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 7; 2025: 6; 2026: 3.

16 papers.

Sort:

By area:

  1. 2024GraphLoRA: Structure-Aware Contrastive Low-Rank Adaptation for Cross-Graph Transfer Learningcited by 21
  2. 2024RAEmoLLM: Retrieval Augmented LLMs for Cross-Domain Misinformation Detection Using In-Context Learning Based on Emotional Informationcited by 14
  3. 2024Retrieval-Augmented Data Augmentation for Low-Resource Domain Taskscited by 12
  4. 2025LLLMs: A Data-Driven Survey of Evolving Research on Limitations of Large Language Modelscited by 9
  5. 2025Logits are All We Need to Adapt Closed Modelscited by 5
  6. 2024Few-shot Prompting for Pairwise Ranking: An Effective Non-Parametric Retrieval Modelcited by 4
  7. 2025Unlocking In-Context Learning for Natural Datasets Beyond Language Modellingcited by 4
  8. 2024Controlling Out-of-Domain Gaps in LLMs for Genre Classification and Generated Text Detectioncited by 3
  9. 2025Efficient Domain-adaptive Continual Pretraining for the Process Industry in the German Languagecited by 3
  10. 2024Towards Robustness of Text-to-Visualization Translation against Lexical and Phrasal Variabilitycited by 2
  11. 2025K-COMP: Retrieval-Augmented Medical Domain Question Answering With Knowledge-Injected Compressorcited by 2
  12. 2026Learning to Query History: Nonstationary Classification via Learned Retrievalcited by 1
4 more
  1. 2024Depth F1: Improving Evaluation of Cross-Domain Text Classification by Measuring Semantic Generalizabilitycited by 0
  2. 2025XGen-Q: An Explainable Domain-Adaptive LLM Framework with Retrieval-Augmented Generation for Software Securitycited by 0
  3. 2026Beyond Distribution Estimation: Simplex Anchored Structural Inference Towards Universal Semi-Supervised Learningcited by 0
  4. 2026Retrieval-Augmented Generation to Support Railways Engineering Tasks: A Case Studycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Similarity-Based Domain Adaptation with LLMs
  2. 2026CoDA: Towards Effective Cross-domain Knowledge Transfer via CoT-guided Domain Adaptation
  3. 2023kNN-Adapter: Efficient Domain Adaptation for Black-Box Language Models
  4. 2024UCDR-Adapter: Exploring Adaptation of Pre-Trained Vision-Language Models for Universal Cross-Domain Retrieval
  5. 2024Improving Few-Shot Cross-Domain Named Entity Recognition by Instruction Tuning a Word-Embedding based Retrieval Augmented Large Language Model
  6. 2024RARe: Retrieval Augmented Retrieval with In-Context Examples
  7. 2026LFM: Leveraging Foundation Models for Source-Free Universal Domain Adaptation
  8. 2024Efficient LLM Context Distillation