mlsanityData to 7 Oct 2026

Augmentation-Adapted Retriever Improves Generalization of Language Models as Generic Plug-In

arXiv 2305.17331 ↗First posted 27 May 2023cs.CL cs.LGcited by 71

Not on the map. Why?

Cited by 41 in Retrieval-augmented generation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Retrieval augmentation can aid language models (LMs) in knowledge-intensive tasks by supplying them with external information. Prior works on retrieval augmentation usually jointly fine-tune the retriever and the LM, making them closely coupled. In this paper, we explore the scheme of generic retrieval plug-in: the retriever is to assist target LMs that may not be known beforehand or are unable to be fine-tuned together. To retrieve useful documents for unseen target LMs, we propose augmentation-adapted retriever (AAR), which learns LM's preferences obtained from a known source LM. Experiments on the MMLU and PopQA datasets demonstrate that our AAR trained with a small source LM is able to significantly improve the zero-shot generalization of larger target LMs ranging from 250M Flan-T5 to 175B InstructGPT. Further analysis indicates that the preferences of different LMs overlap, enabling AAR trained with a single source LM to serve as a generic plug-in for various target LMs. Our code is open-sourced at https://github.com/OpenMatch/Augmentation-Adapted-Retriever.

Key facts. Cited by 71 papers in the map (data through 7 Oct 2026). Builds on 40 of its 52 references that are in the map. First posted on arXiv 27 May 2023.

Builds on

40 of 52 references.

References · 40

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2022Training language models to follow instructions with human feedbackcited by 15,712
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  6. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  7. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  8. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  9. 2020Scaling Laws for Neural Language Modelscited by 6,411
  10. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  11. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  12. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
28 more
  1. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  2. 2017Billion-scale similarity search with GPUscited by 2,477
  3. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  4. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
  5. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  6. 2021Improving language models by retrieving from trillions of tokenscited by 1,278
  7. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
  8. 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,242
  9. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
  10. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
  11. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 940
  12. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 846
  13. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 786
  14. 2022Galactica: A Large Language Model for Sciencecited by 736
  15. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 713
  16. 2021Learning To Retrieve Prompts for In-Context Learningcited by 641
  17. 2020KILT: a Benchmark for Knowledge Intensive Language Taskscited by 571
  18. 2023A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPTcited by 364
  19. 2022Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLPcited by 316
  20. 2022Generate rather than Retrieve: Large Language Models are Strong Context Generatorscited by 296
  21. 2022OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalizationcited by 235
  22. 2020Distilling Knowledge from Reader to Retriever for Question Answeringcited by 226
  23. 2022Rethinking with Retrieval: Faithful Large Language Model Inferencecited by 145
  24. 2022Task-aware Retrieval with Instructionscited by 107
  25. 2022Unsupervised Cross-Task Generalization via Retrieval Augmentationcited by 42
  26. 2021Zero-Shot Dense Retrieval with Momentum Adversarial Domain Invariant Representationscited by 27
  27. 2022Grape: Knowledge Graph Enhanced Passage Reader for Open-domain Question Answeringcited by 22
  28. 2023Augmenting Zero-Shot Dense Retrievers with Plug-in Mixture-of-Memoriescited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 8; 2024: 33; 2025: 24; 2026: 6.

71 papers.

Sort:

By area:

  1. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  2. 2025A-MEM: Agentic Memory for LLM Agentscited by 1,000
  3. 2024A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Modelscited by 450
  4. 2024VisRAG: Vision-based Retrieval-augmented Generation on Multi-modality Documentscited by 237
  5. 2024FlashRAG: A Modular Toolkit for Efficient Retrieval-Augmented Generation Researchcited by 165
  6. 2024From Matching to Generation: A Survey on Generative Information Retrievalcited by 89
  7. 2024A Survey on Retrieval-Augmented Text Generation for Large Language Modelscited by 77
  8. 2023A Multi-Task Embedder For Retrieval Augmented LLMscited by 63
  9. 2025RAG vs. GraphRAG: A Systematic Evaluation and Key Insightscited by 51
  10. 2024Modular RAG: Transforming RAG Systems into LEGO-like Reconfigurable Frameworkscited by 41
  11. 2024UDA: A Benchmark Suite for Retrieval Augmented Generation in Real-world Document Analysiscited by 40
  12. 2024RAGEval: Scenario Specific RAG Evaluation Dataset Generation Frameworkcited by 39
38 more
  1. 2025A Survey on Knowledge-Oriented Retrieval-Augmented Generationcited by 34
  2. 2023How Do Large Language Models Capture the Ever-changing World Knowledge? A Review of Recent Advancescited by 33
  3. 2024RAG-DDR: Optimizing Retrieval-Augmented Generation Using Differentiable Data Rewardscited by 33
  4. 2023PDFTriage: Question Answering over Long, Structured Documentscited by 29
  5. 2024Mask-based Membership Inference Attacks for Retrieval-Augmented Generationcited by 29
  6. 2023Trends in Integration of Knowledge and Large Language Models: A Survey and Taxonomy of Methods, Benchmarks, and Applicationscited by 28
  7. 2025SePer: Measure Retrieval Utility Through The Lens Of Semantic Perplexity Reductioncited by 24
  8. 2024Fact-Aware Multimodal Retrieval Augmentation for Accurate Medical Radiology Report Generationcited by 23
  9. 2023Towards Verifiable Text Generation with Evolving Memory and Self-Reflectioncited by 21
  10. 2024FunnelRAG: A Coarse-to-Fine Progressive Retrieval Paradigm for RAGcited by 18
  11. 2025Benchmarking Poisoning Attacks against Retrieval-Augmented Generationcited by 17
  12. 2024RichRAG: Crafting Rich Responses for Multi-faceted Queries in Retrieval-Augmented Generationcited by 16
  13. 2024DR-RAG: Applying Dynamic Document Relevance to Retrieval-Augmented Generation for Question-Answeringcited by 14
  14. 2024RAG based Question-Answering for Contextual Response Prediction Systemcited by 14
  15. 2024FIT-RAG: Black-Box RAG with Factual Information and Token Reductioncited by 13
  16. 2024Bridging Relevance and Reasoning: Rationale Distillation in Retrieval-Augmented Generationcited by 13
  17. 2025Training a Utility-based Retriever Through Shared Context Attribution for Retrieval-Augmented Language Modelscited by 13
  18. 2024AesopAgent: Agent-driven Evolutionary System on Story-to-Video Productioncited by 12
  19. 2024Graph of Records: Boosting Retrieval Augmented Generation for Long-context Summarization with Graphscited by 12
  20. 2025Benchmarking Retrieval-Augmented Generation in Multi-Modal Contextscited by 12
  21. 2025Who Taught the Lie? Responsibility Attribution for Poisoned Knowledge in Retrieval-Augmented Generationcited by 12
  22. 2024ARL2: Aligning Retrievers for Black-box Large Language Models via Self-guided Adaptive Relevance Labelingcited by 10
  23. 2024Say More with Less: Understanding Prompt Learning Behaviors through Gist Compressioncited by 8
  24. 2024Pistis-RAG: Enhancing Retrieval-Augmented Generation with Human Feedbackcited by 8
  25. 2025Bring Your Own Knowledge: A Survey of Methods for LLM Knowledge Expansioncited by 8
  26. 2025RegionRAG: Region-level Retrieval-Augmented Generation for Visual Document Understandingcited by 8
  27. 2024MURRE: Multi-Hop Table Retrieval with Removal for Open-Domain Text-to-SQLcited by 7
  28. 2024W-RAG: Weakly Supervised Dense Retrieval in RAG for Open-domain Question Answeringcited by 6
  29. 2024DocTabQA: Answering Questions from Long Documents Using Tablescited by 6
  30. 2025UltraRAG: A Modular and Automated Toolkit for Adaptive Retrieval-Augmented Generationcited by 6
  31. 2025HeteRAG: A Heterogeneous Retrieval-augmented Generation Framework with Decoupled Knowledge Representationscited by 6
  32. 2025RAVine: Reality-Aligned Evaluation for Agentic Searchcited by 6
  33. 2024QRMeM: Unleash the Length Limitation through Question then Reflection Memory Mechanismcited by 5
  34. 2024Context-Driven Index Trimming: A Data Quality Perspective to Enhancing Precision of RALMscited by 5
  35. 2025Evaluating and Improving Robustness in Large Language Models: A Survey and Future Directionscited by 5
  36. 2023Defining a New NLP Playgroundcited by 4
  37. 2024Redefining Information Retrieval of Structured Database via Large Language Modelscited by 4
  38. 2024ONSEP: A Novel Online Neural-Symbolic Framework for Event Prediction Based on Large Language Modelcited by 4

50 most cited of 71 All 71 (JSON)