mlsanityData to 7 Oct 2026

Retrieval Augmented Code Generation and Summarization

arXiv 2108.11601 ↗First posted 26 Aug 2021cs.SE cs.CLcited by 148

On the map: Retrieval-augmented generation → RAG for code

Abstract (arXiv)

Software developers write a lot of source code and documentation during software development. Intrinsically, developers often recall parts of source code or code summaries that they had written in the past while implementing software or documenting them. To mimic developers' code or summary generation behavior, we propose a retrieval augmented framework, REDCODER, that retrieves relevant code or summaries from a retrieval database and provides them as a supplement to code generation or summarization models. REDCODER has a couple of uniqueness. First, it extends the state-of-the-art dense retrieval technique to search for relevant code or summaries. Second, it can work with retrieval databases that include unimodal (only code or natural language description) or bimodal instances (code-description pairs). We conduct experiments and extensive analysis on two benchmark datasets of code generation and summarization in Java and Python, and the promising results endorse the effectiveness of our proposed retrieval augmented framework.

Key facts. Cited by 148 papers in the map (data through 7 Oct 2026). Placed in RAG for code, one of 15 directions in Retrieval-augmented generation. Builds on 38 of its 60 references that are in the map. First posted on arXiv 26 Aug 2021.

Builds on

38 of 60 references.

Closest ancestors · 1

Same direction, newest first

  1. 2020Retrieval-Augmented Generation for Code Summarization via Hybrid GNN9 Jun

Other references · 37

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  5. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  6. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  7. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,286
  8. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
  9. 2015Effective Approaches to Attention-based Neural Machine Translationcited by 1,024
  10. 2021CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generationcited by 732
  11. 2019CodeSearchNet Challenge: Evaluating the State of Semantic Code Searchcited by 638
  12. 2020GraphCodeBERT: Pre-training Code Representations with Data Flowcited by 600
25 more
  1. 2020CodeBLEU: a Method for Automatic Evaluation of Code Synthesiscited by 468
  2. 2021Unified Pre-training for Program Understanding and Generationcited by 361
  3. 2017A Survey of Machine Learning for Big Code and Naturalnesscited by 220
  4. 2019Poly-encoders: Transformer Architectures and Pre-training Strategies for Fast and Accurate Multi-sentence Scoringcited by 164
  5. 2020A Transformer-based Approach for Source Code Summarizationcited by 129
  6. 2019Learning Dense Representations for Entity Retrievalcited by 122
  7. 2018Mapping Language to Code in Programmatic Contextcited by 104
  8. 2017Abstract Syntax Networks for Code Generation and Semantic Parsingcited by 92
  9. 2016A Convolutional Attention Network for Extreme Summarization of Source Codecited by 90
  10. 2019A Neural Model for Generating Natural Language Summaries of Program Subroutinescited by 76
  11. 2018A Retrieve-and-Edit Framework for Predicting Structured Outputscited by 73
  12. 2019Code Generation as a Dual Task of Code Summarizationcited by 62
  13. 2016Deep API Learningcited by 41
  14. 2018Retrieval-Based Neural Code Generationcited by 41
  15. 2021In-IDE Code Generation from Natural Language: Promise and Challengescited by 41
  16. 2020Incorporating External Knowledge through Pre-training for Natural Language to Code Generationcited by 36
  17. 2015Quantization based Fast Inner Product Searchcited by 29
  18. 2019Automatic Source Code Summarization with Extended Tree-LSTMcited by 28
  19. 2019Tree-Transformer: A Transformer-Based Method for Correction of Tree-Structured Datacited by 20
  20. 2018Automatic Generation of Text Descriptive Comments for Code Blockscited by 18
  21. 2019Coupling Retrieval and Meta-Learning for Context-Dependent Semantic Parsingcited by 18
  22. 2019Learning Programmatic Idioms for Scalable Semantic Parsingcited by 17
  23. 2018Building Language Models for Text with Named Entitiescited by 15
  24. 2018Evaluating How Developers Use General-Purpose Web-Search for Code Retrievalcited by 6
  25. 2016Software search is not a science, even among scientists: A survey of how scientists and engineers find softwarecited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 14; 2023: 23; 2024: 46; 2025: 40; 2026: 24.

148 papers.

Sort:

By area:

  1. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
  2. 2024A Survey on Large Language Models for Code Generationcited by 646
  3. 2024A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Modelscited by 450
  4. 2023LeanDojo: Theorem Proving with Retrieval-Augmented Language Modelscited by 414
  5. 2023RepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generationcited by 357
  6. 2024Retrieval-Augmented Generation for AI-Generated Content: A Surveycited by 352
  7. 2023CodeT5+: Open Code Large Language Models for Code Understanding and Generationcited by 327
  8. 2024MapCoder: Multi-Agent Code Generation for Competitive Problem Solvingcited by 177
  9. 2024The Good and The Bad: Exploring Privacy Issues in Retrieval-Augmented Generation (RAG)cited by 159
  10. 2022DocPrompting: Generating Code by Retrieving the Docscited by 151
  11. 2023Physics of Language Models: Part 3.2, Knowledge Manipulationcited by 142
  12. 2022ReACC: A Retrieval-Augmented Code Completion Frameworkcited by 117
38 more
  1. 2023Lift Yourself Up: Retrieval-augmented Text Generation with Self Memorycited by 103
  2. 2024A Comprehensive Survey on Process-Oriented Automatic Text Summarization with Exploration of LLM-Based Methodscited by 101
  3. 2023Retrieving Multimodal Information for Augmented Generation: A Surveycited by 84
  4. 2024BadRAG: Identifying Vulnerabilities in Retrieval Augmented Generation of Large Language Modelscited by 84
  5. 2024A Systematic Survey and Critical Review on Evaluating Large Language Models: Challenges, Limitations, and Recommendationscited by 84
  6. 2023Is ChatGPT the Ultimate Programming Assistant -- How far is it?cited by 72
  7. 2023Enhancing Financial Sentiment Analysis via Retrieval Augmented Large Language Modelscited by 67
  8. 2024LitLLM: A Toolkit for Scientific Literature Reviewcited by 61
  9. 2024RAGChecker: A Fine-grained Framework for Diagnosing Retrieval-Augmented Generationcited by 59
  10. 2025VDocRAG: Retrieval-Augmented Generation over Visually-Rich Documentscited by 57
  11. 2022When Language Model Meets Private Librarycited by 54
  12. 2022NatGen: Generative pre-training by "Naturalizing" source codecited by 53
  13. 2024Mitigating the Privacy Issues in Retrieval-Augmented Generation (RAG) via Pure Synthetic Datacited by 52
  14. 2022Compilable Neural Code Generation with Compiler Feedbackcited by 46
  15. 2023xCodeEval: A Large Scale Multilingual Multitask Benchmark for Code Understanding, Generation, Translation and Retrievalcited by 46
  16. 2021Multilingual training for Software Engineeringcited by 43
  17. 2023SelfEvolve: A Code Evolution Framework via Large Language Modelscited by 43
  18. 2023RAP-Gen: Retrieval-Augmented Patch Generation with CodeT5 for Automatic Program Repaircited by 43
  19. 2025Ask in Any Modality: A Comprehensive Survey on Multimodal Retrieval-Augmented Generationcited by 42
  20. 2023RepoFusion: Training Code Models to Understand Your Repositorycited by 40
  21. 2024Dataflow-Guided Retrieval Augmentation for Repository-Level Code Completioncited by 40
  22. 2024GraphCoder: Enhancing Repository-Level Code Completion via Code Context Graph-based Retrieval and Language Modelcited by 40
  23. 2024Automatic Programming: Large Language Models and Beyondcited by 35
  24. 2024RAG-DDR: Optimizing Retrieval-Augmented Generation Using Differentiable Data Rewardscited by 33
  25. 2024CodeXEmbed: A Generalist Embedding Model Family for Multiligual and Multi-task Code Retrievalcited by 33
  26. 2025CODESIM: Multi-Agent Code Generation and Problem Solving through Simulation-Driven Planning and Debuggingcited by 31
  27. 2025A Systematic Review of Key Retrieval-Augmented Generation (RAG) Systems: Progress, Gaps, and Future Directionscited by 30
  28. 2023Exploring Parameter-Efficient Fine-Tuning Techniques for Code Generation with Large Language Modelscited by 28
  29. 2023AceCoder: Utilizing Existing Code to Enhance Code Generationcited by 25
  30. 2024RAG and RAU: A Survey on Retrieval-Augmented Language Model in Natural Language Processingcited by 25
  31. 2024Rango: Adaptive Retrieval-Augmented Proving for Automated Software Verificationcited by 25
  32. 2024CodeMirage: Hallucinations in Code Generated by Large Language Modelscited by 23
  33. 2024EVOR: Evolving Retrieval for Code Generationcited by 22
  34. 2025What to Retrieve for Effective Retrieval-Augmented Code Generation? An Empirical Study and Beyondcited by 22
  35. 2022CodeRetriever: Unimodal and Bimodal Contrastive Learning for Code Searchcited by 21
  36. 2022CoNT: Contrastive Neural Text Generationcited by 21
  37. 2023CodeTF: One-stop Transformer Library for State-of-the-art Code LLMscited by 21
  38. 2024RAM-EHR: Retrieval Augmentation Meets Clinical Predictions on Electronic Health Recordscited by 21

50 most cited of 148 All 148 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Retrieval-Augmented Code Generation: A Survey with Focus on Repository-Level Approaches
  2. 2024CodeRAG-Bench: Can Retrieval Augment Code Generation?
  3. 2025A Deep Dive into Retrieval-Augmented Generation for Code Completion: Experience on WeChat
  4. 2026Do Not Treat Code as Natural Language: Implications for Repository-Level Code Generation and Beyond
  5. 2026CodeMMR: Bridging Natural Language, Code, and Image for Unified Retrieval
  6. 2025When Retriever Meets Generator: A Joint Model for Code Comment Generation ↗
  7. 2025Impact-driven Context Filtering For Cross-file Code Completion
  8. 2025cAST: Enhancing Code Retrieval-Augmented Generation with Structural Chunking via Abstract Syntax Tree