mlsanityData to 14 Oct 2026

RAP-Gen: Retrieval-Augmented Patch Generation with CodeT5 for Automatic Program Repair

arXiv 2309.06057 ↗First posted 12 Sep 2023cs.SE cs.CLcited by 44

On the map: Retrieval-augmented generation → RAG for code

Abstract (arXiv)

Automatic program repair (APR) is crucial to reduce manual debugging efforts for developers and improve software reliability. While conventional search-based techniques typically rely on heuristic rules or a redundancy assumption to mine fix patterns, recent years have witnessed the surge of deep learning (DL) based approaches to automate the program repair process in a data-driven manner. However, their performance is often limited by a fixed set of parameters to model the highly complex search space of APR. To ease such burden on the parametric models, in this work, we propose a novel Retrieval-Augmented Patch Generation framework (RAP-Gen) by explicitly leveraging relevant fix patterns retrieved from a codebase of previous bug-fix pairs. Specifically, we build a hybrid patch retriever to account for both lexical and semantic matching based on the raw source code in a language-agnostic manner, which does not rely on any code-specific features. In addition, we adapt a code-aware language model CodeT5 as our foundation model to facilitate both patch retrieval and generation tasks in a unified manner. We adopt a stage-wise approach where the patch retriever first retrieves a relevant external bug-fix pair to augment the buggy input for the CodeT5 patch generator, which synthesizes a ranked list of repair patch candidates. Notably, RAP-Gen is a generic APR framework that can flexibly integrate different patch retrievers and generators to repair various types of bugs. We thoroughly evaluate RAP-Gen on three benchmarks in two programming languages, including the TFix benchmark in JavaScript, and Code Refinement and Defects4J benchmarks in Java, where the bug localization information may or may not be provided. Experimental results show that RAP-Gen significantly outperforms previous state-of-the-art approaches on all benchmarks, e.g., repairing 15 more bugs on 818 Defects4J bugs.

Key facts. Cited by 44 papers in the map (data through 14 Oct 2026). Placed in RAG for code, one of 15 directions in Retrieval-augmented generation. Builds on 43 of its 82 references that are in the map. First posted on arXiv 12 Sep 2023.

Builds on

43 of 82 references.

Closest ancestors · 2

Same direction, newest first

  1. 2022ReACC: A Retrieval-Augmented Code Completion Framework15 Mar
  2. 2021Retrieval Augmented Code Generation and Summarization26 Aug

Other references · 41

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  6. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  7. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  8. 2014Sequence to Sequence Learning with Neural Networkscited by 2,986
  9. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  10. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,310
  11. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,210
  12. 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 906
29 more
  1. 2021CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generationcited by 744
  2. 2020GraphCodeBERT: Pre-training Code Representations with Data Flowcited by 609
  3. 2017Convolutional Sequence to Sequence Learningcited by 552
  4. 2021Unified Pre-training for Program Understanding and Generationcited by 366
  5. 2022CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learningcited by 358
  6. 2023CodeT5+: Open Code Large Language Models for Code Understanding and Generationcited by 333
  7. 2015Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networkscited by 306
  8. 2022A Survey on Retrieval-Augmented Text Generationcited by 158
  9. 2018SequenceR: Sequence-to-Sequence Learning for End-to-End Program Repaircited by 145
  10. 2018An Empirical Study on Learning Bug-Fixing Patches in the Wild via Neural Machine Translationcited by 108
  11. 2021CURE: Code-Aware Neural Machine Translation for Automatic Program Repaircited by 86
  12. 2018A Retrieve-and-Edit Framework for Predicting Structured Outputscited by 73
  13. 2022Repair Is Nearly Generation: Multilingual Program Repair with LLMscited by 68
  14. 2021CoTexT: Multi-task Learning with Code-Text Transformercited by 61
  15. 2021Self-Supervised Bug Detection and Repaircited by 59
  16. 2021A Syntax-Guided Edit Decoder for Neural Program Repaircited by 57
  17. 2021Neural Program Repair with Execution-based Backpropagationcited by 50
  18. 2017Code Completion with Neural Attention and Pointer Networkscited by 49
  19. 2022SelfAPR: Self-supervised Program Repair with Test Execution Diagnosticscited by 33
  20. 2017Sorting and Transforming Program Repair Ingredients via Deep Learning Code Similaritiescited by 26
  21. 2017ARJA: Automated Repair of Java Programs via Multi-Objective Genetic Programmingcited by 26
  22. 2022DEAR: A Novel Deep Learning-based Approach for Automated Program Repaircited by 22
  23. 2023EditSum: A Retrieve-and-Edit Framework for Source Code Summarizationcited by 18
  24. 2018How to Design a Program Repair Bot? Insights from the Repairnator Projectcited by 11
  25. 2022Detect-Localize-Repair: A Unified Framework for Learning to Debug with CodeT5cited by 10
  26. 2022Fix Bugs with Transformer through a Neural-Symbolic Edit Grammarcited by 8
  27. 2019Repairnator patches programs automaticallycited by 3
  28. 2014Do the Fix Ingredients Already Exist? An Empirical Inquiry into the Redundancy Assumptions of Program Repair Approachescited by 2
  29. 2018Production-Driven Patch Generationcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 12; 2025: 22; 2026: 8.

44 papers.

Sort:

By area:

  1. 2024Retrieval-Augmented Generation for AI-Generated Content: A Surveycited by 354
  2. 2024AutoCodeRover: Autonomous Program Improvementcited by 278
  3. 2024MAGIS: LLM-Based Multi-Agent Framework for GitHub Issue Resolutioncited by 134
  4. 2024Large Language Models for Cyber Security: A Systematic Literature Reviewcited by 68
  5. 2024Vul-RAG: Enhancing LLM-based Vulnerability Detection via Knowledge-level RAGcited by 68
  6. 2023RepairLLaMA: Efficient Representations and Fine-Tuned Adapters for Program Repaircited by 50
  7. 2025A Survey on Knowledge-Oriented Retrieval-Augmented Generationcited by 36
  8. 2024Rango: Adaptive Retrieval-Augmented Proving for Automated Software Verificationcited by 27
  9. 2024How secure is AI-generated Code: A Large-Scale Comparison of Large Language Modelscited by 24
  10. 2025Retrieval-Augmented Code Generation: A Survey with Focus on Repository-Level Approachescited by 23
  11. 2024EVOR: Evolving Retrieval for Code Generationcited by 22
  12. 2025SweRank: Software Issue Localization with Code Rankingcited by 20
32 more
  1. 2024Are LLMs Correctly Integrated into Software Systems?cited by 15
  2. 2024Aligning the Objective of LLM-based Program Repaircited by 12
  3. 2025Exploring the Security Threats of Knowledge Base Poisoning in Retrieval-Augmented Code Generationcited by 9
  4. 2024Retrieval-Augmented Test Generation: How Far Are We?cited by 7
  5. 2023Out of Context: How important is Local Context in Neural Program Repair?cited by 5
  6. 2025PredicateFix: Repairing Static Analysis Alerts with Bridging Predicatescited by 5
  7. 2025RAG or Fine-tuning? A Comparative Study on LCMs-based Code Completion in Industrycited by 5
  8. 2025Generative AI for Self-Adaptive Systems: State of the Art and Research Roadmapcited by 5
  9. 2025Improving Deep Assertion Generation via Fine-Tuning Retrieval-Augmented Pre-trained Language Modelscited by 4
  10. 2025REIS: A High-Performance and Energy-Efficient Retrieval System with In-Storage Processingcited by 4
  11. 2025Unlocking LLM Repair Capabilities Through Cross-Language Translation and Multi-Agent Refinementcited by 3
  12. 2025Enhancing Code Generation via Bidirectional Comment-Level Mutual Groundingcited by 3
  13. 2025Across Programming Language Silos: A Study on Cross-Lingual Retrieval-augmented Code Generationcited by 3
  14. 2025Practical Code RAG at Scale: Task-Aware Retrieval Design Choices under Compute Budgetscited by 3
  15. 2026Learning When to Remember: Risk-Sensitive Contextual Bandits for Abstention-Aware Memory Retrieval in LLM-Based Coding Agentscited by 3
  16. 2024Automated Code Fix Suggestions for Accessibility Issues in Mobile Appscited by 2
  17. 2025Improving Retrieval-Augmented Deep Assertion Generation via Joint Trainingcited by 2
  18. 2026FailureMem: A Failure-Aware Multimodal Framework for Autonomous Software Repaircited by 2
  19. 2025Accelerating Automatic Program Repair with Dual Retrieval-Augmented Fine-Tuning and Patch Generation on Large Language Modelscited by 1
  20. 2025Collaborative Agents for Automated Program Repair in Rubycited by 1
  21. 2025From LLMs to Agents: A Comparative Evaluation of LLMs and LLM-based Agents in Security Patch Detectioncited by 1
  22. 2025Exploring the Security Threats of Retriever Backdoors in Retrieval-Augmented Code Generationcited by 1
  23. 2026A Study on the Impact of Fault localization Granularity for Repository-Scale Code Repair Taskscited by 1
  24. 2026RAVEN: Agentic RAG for Automated Vulnerability Repaircited by 1
  25. 2026CodePoisonRAG: Knowledge Poisoning Attacks on Retrieval-Augmented Code Generationcited by 1
  26. 2025Self-Bootstrapping Automated Program Repair: Using LLMs to Generate and Evaluate Synthetic Training Data for Bug Repaircited by 0
  27. 2025T3: Multi-level Tree-based Automatic Program Repair with Large Language Modelscited by 0
  28. 2025CoRaCMG: Contextual Retrieval-Augmented Framework for Commit Message Generationcited by 0
  29. 2025GrowthHacker: Automated Off-Policy Evaluation Optimization Using Code-Modifying LLM Agentscited by 0
  30. 2026On the Impact of Code Comments for Automated Bug-Fixing: An Empirical Studycited by 0
  31. 2026PYTHALAB-MERA: Validation-Grounded Memory, Retrieval, and Acceptance Control for Frozen-LLM Coding Agentscited by 0
  32. 2026RETRACE: From Entangled Repair Histories to Reusable Experience for CI Repaircited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025MultiMend: Multilingual Program Repair with Context Augmentation and Multi-Hunk Patch Generation
  2. 2025The Art of Repair: Optimizing Iterative Program Repair with Instruction-Tuned Models
  3. 2025REFINE: Enhancing Program Repair Agents through Context-Aware Patch Refinement
  4. 2023KNOD: Domain Knowledge Distilled Tree Decoder for Automated Program Repair
  5. 2025ReCode: Improving LLM-based Code Repair with Fine-Grained Retrieval-Augmented Generation
  6. 2025Automated Repair of C Programs Using Large Language Models
  7. 2026Refine After Generation: Toward Correct and Concise Patches in LLM-based Program Repair
  8. 2026Rethinking Automated Program Repair: The Impact of Bug Complexity, Fault Localization, and LLM Cost-efficiency