A Systematic Literature Review on Large Language Models for Automated Program Repair
Not on the map. Why?
Abstract (arXiv)
Automated Program Repair (APR) attempts to patch software bugs and reduce manual debugging efforts. Very recently, with the advances in Large Language Models (LLMs), an increasing number of APR techniques have been proposed, facilitating software development and maintenance and demonstrating remarkable performance. However, due to ongoing explorations in the LLM-based APR field, it is challenging for researchers to understand the current achievements, challenges, and potential opportunities. This work provides the first systematic literature review to summarize the applications of LLMs in APR between 2020 and 2025. We analyze 189 relevant papers from LLMs, APR and their integration perspectives. First, we categorize existing popular LLMs that are applied to support APR and outline four types of utilization strategies for their deployment. Besides, we detail some specific repair scenarios that benefit from LLMs, e.g., semantic bugs and security vulnerabilities. Furthermore, we discuss several critical aspects of integrating LLMs into APR research, e.g., input forms and open science. Finally, we highlight a set of challenges remaining to be investigated and the potential guidelines for future research. Overall, our paper provides a systematic overview of the research landscape to the APR community, helping researchers gain a comprehensive understanding of achievements and promote future research.
Key facts. Cited by 37 papers in the map (data through 7 Oct 2026). First posted on arXiv 2 May 2024.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2024: 9; 2025: 11; 2026: 17.
37 papers.
Sort:
By area:
- 2024Agentless: Demystifying LLM-based Software Engineering Agentscited by 400
- 2024When LLMs Meet Cybersecurity: A Systematic Literature Reviewcited by 70
- 2025Type-Constrained Code Generation with Language Modelscited by 32
- 2024Source Code Summarization in the Era of Large Language Modelscited by 30
- 2025AI Agentic Programming: A Survey of Techniques, Challenges, and Opportunitiescited by 26
- 2025debug-gym: A Text-Based Environment for Interactive Debuggingcited by 19
- 2025Afterburner: Reinforcement Learning Facilitates Self-Improving Code Efficiency Optimizationcited by 14
- 2025Seeing is Fixing: Cross-Modal Reasoning with Multimodal LLMs for Visual Software Issue Fixingcited by 11
- 2024RepairBench: Leaderboard of Frontier Models for Program Repaircited by 9
- 2024Can GPT-O1 Kill All Bugs? An Evaluation of GPT-Family LLMs on QuixBugscited by 7
- 2024ESALE: Enhancing Code-Summary Alignment Learning for Source Code Summarizationcited by 6
- 2025Agentic Program Repair from Test Failures at Scale: A Neuro-symbolic approach with static analysis and test execution feedbackcited by 6
25 more
- 2025Can LLMs Find Bugs in Code? An Evaluation from Beginner Errors to Security Vulnerabilities in Python and C++cited by 3
- 2026CktEvo: Repository-Level RTL Code Benchmark for Design Evolutioncited by 3
- 2024Automated Code Fix Suggestions for Accessibility Issues in Mobile Appscited by 2
- 2024An Empirical Study on Self-correcting Large Language Models for Data Science Code Generationcited by 2
- 2024Detection Made Easy: Potentials of Large Language Models for Solidity Vulnerabilitiescited by 2
- 2025Improving Retrieval-Augmented Deep Assertion Generation via Joint Trainingcited by 2
- 2025HAFixAgent: History-Aware Program Repair Agentcited by 2
- 2026Polaris: A G"odel Agent Framework for Small Language Models through Experience-Abstracted Policy Repaircited by 2
- 2026LLM-Based Multi-Agent Systems for Code Generation: A Multi-Vocal Literature Reviewcited by 2
- 2026ARISE: A Repository-level Graph Representation and Toolset for Agentic Program Repair and Fault Localizationcited by 2
- 2026DeepDiscovery: A Location-Inference Framework for Task-Level Repository Understandingcited by 1
- 2026Writing Bug Reports for Software Repair Agents: What Information Matters Most?cited by 1
- 2026Multi-Perspective Agentic Program Repair via Code Property Graphs and Temporal Execution Graphscited by 1
- 2025T3: Multi-level Tree-based Automatic Program Repair with Large Language Modelscited by 0
- 2025Learning Bug Context for PyTorch-to-JAX Translation with LLMscited by 0
- 2026A Metamorphic Testing Approach to Diagnosing Memorization in LLM-Based Program Repaircited by 0
- 2026An Empirical Evaluation of Locally Deployed LLMs for Bug Detection in Python Codecited by 0
- 2026BLAgent: Agentic RAG for File-Level Bug Localizationcited by 0
- 2026Benchmarking Large Language Models on Floating-Point Error Classificationcited by 0
- 2026Beyond Textual Repository Exploration: Dual-Modal Structural Reasoning for Agentic Issue Resolutioncited by 0
- 2026Multi-Agent LLM Collaboration for Unit Test Generation via Human-Testing-Inspired Workflowscited by 0
- 2026Retrieval-Oriented Code Representations in Agentic Bug Localizationcited by 0
- 2026Deep Academic Survey: Stateful Agentic Closed-Loop Paradigm for Academic Survey Automationcited by 0
- 2026The Vibe Shift in Software Engineering: Evaluating AI-Led Conversational Programming for Performance, Cognition, and Responsible Adoptioncited by 0
- 2026Test-Driven Approaches to Software Engineering with Large Language Models: A Survey of Phases, Tasks, and Agent Skillscited by 0
- 2026Test-Driven Approaches to Software Engineering with Large Language Models: A Survey of Phases, Tasks, and Agent Skills10 Sep 2026
- 2026The Vibe Shift in Software Engineering: Evaluating AI-Led Conversational Programming for Performance, Cognition, and Responsible Adoption9 Sep 2026
- 2026Deep Academic Survey: Stateful Agentic Closed-Loop Paradigm for Academic Survey Automation18 Aug 2026
- 2026Multi-Perspective Agentic Program Repair via Code Property Graphs and Temporal Execution Graphs14 Jul 2026
- 2026Retrieval-Oriented Code Representations in Agentic Bug Localization13 Jul 2026
- 2026Writing Bug Reports for Software Repair Agents: What Information Matters Most?10 Jul 2026
- 2026Multi-Agent LLM Collaboration for Unit Test Generation via Human-Testing-Inspired Workflows10 Jul 2026
- 2026Beyond Textual Repository Exploration: Dual-Modal Structural Reasoning for Agentic Issue Resolution2 Jul 2026
- 2026Benchmarking Large Language Models on Floating-Point Error Classification30 Jun 2026
- 2026DeepDiscovery: A Location-Inference Framework for Task-Level Repository Understanding22 Jun 2026
- 2026BLAgent: Agentic RAG for File-Level Bug Localization18 May 2026
- 2026ARISE: A Repository-level Graph Representation and Toolset for Agentic Program Repair and Fault Localization4 May 2026
25 more
- 2026An Empirical Evaluation of Locally Deployed LLMs for Bug Detection in Python Code25 Apr 2026
- 2026A Metamorphic Testing Approach to Diagnosing Memorization in LLM-Based Program Repair23 Apr 2026
- 2026Polaris: A G"odel Agent Framework for Small Language Models through Experience-Abstracted Policy Repair24 Mar 2026
- 2026LLM-Based Multi-Agent Systems for Code Generation: A Multi-Vocal Literature Review25 Feb 2026
- 2026CktEvo: Repository-Level RTL Code Benchmark for Design Evolution10 Feb 2026
- 2025HAFixAgent: History-Aware Program Repair Agent2 Nov 2025
- 2025Learning Bug Context for PyTorch-to-JAX Translation with LLMs10 Oct 2025
- 2025Can LLMs Find Bugs in Code? An Evaluation from Beginner Errors to Security Vulnerabilities in Python and C++22 Aug 2025
- 2025AI Agentic Programming: A Survey of Techniques, Challenges, and Opportunities15 Aug 2025
- 2025Agentic Program Repair from Test Failures at Scale: A Neuro-symbolic approach with static analysis and test execution feedback24 Jul 2025
- 2025T3: Multi-level Tree-based Automatic Program Repair with Large Language Models26 Jun 2025
- 2025Seeing is Fixing: Cross-Modal Reasoning with Multimodal LLMs for Visual Software Issue Fixing19 Jun 2025
- 2025Afterburner: Reinforcement Learning Facilitates Self-Improving Code Efficiency Optimization29 May 2025
- 2025Type-Constrained Code Generation with Language Models12 Apr 2025
- 2025debug-gym: A Text-Based Environment for Interactive Debugging27 Mar 2025
- 2025Improving Retrieval-Augmented Deep Assertion Generation via Joint Training15 Feb 2025
- 2024RepairBench: Leaderboard of Frontier Models for Program Repair27 Sep 2024
- 2024Can GPT-O1 Kill All Bugs? An Evaluation of GPT-Family LLMs on QuixBugs16 Sep 2024
- 2024Detection Made Easy: Potentials of Large Language Models for Solidity Vulnerabilities15 Sep 2024
- 2024An Empirical Study on Self-correcting Large Language Models for Data Science Code Generation28 Aug 2024
- 2024Automated Code Fix Suggestions for Accessibility Issues in Mobile Apps7 Aug 2024
- 2024Source Code Summarization in the Era of Large Language Models9 Jul 2024
- 2024ESALE: Enhancing Code-Summary Alignment Learning for Source Code Summarization1 Jul 2024
- 2024Agentless: Demystifying LLM-based Software Engineering Agents1 Jul 2024
- 2024When LLMs Meet Cybersecurity: A Systematic Literature Review6 May 2024
- 2024Agentless: Demystifying LLM-based Software Engineering Agentscited by 400
- 2025AI Agentic Programming: A Survey of Techniques, Challenges, and Opportunitiescited by 26
- 2025debug-gym: A Text-Based Environment for Interactive Debuggingcited by 19
- 2025Agentic Program Repair from Test Failures at Scale: A Neuro-symbolic approach with static analysis and test execution feedbackcited by 6
- 2025HAFixAgent: History-Aware Program Repair Agentcited by 2
- 2026LLM-Based Multi-Agent Systems for Code Generation: A Multi-Vocal Literature Reviewcited by 2
- 2026ARISE: A Repository-level Graph Representation and Toolset for Agentic Program Repair and Fault Localizationcited by 2
- 2026Writing Bug Reports for Software Repair Agents: What Information Matters Most?cited by 1
- 2026Multi-Perspective Agentic Program Repair via Code Property Graphs and Temporal Execution Graphscited by 1
- 2026Beyond Textual Repository Exploration: Dual-Modal Structural Reasoning for Agentic Issue Resolutioncited by 0
- 2026Multi-Agent LLM Collaboration for Unit Test Generation via Human-Testing-Inspired Workflowscited by 0
- 2026Retrieval-Oriented Code Representations in Agentic Bug Localizationcited by 0
12 most cited of 14 All 14 (JSON)
- 2026Test-Driven Approaches to Software Engineering with Large Language Models: A Survey of Phases, Tasks, and Agent Skills10 Sep 2026
- 2026Deep Academic Survey: Stateful Agentic Closed-Loop Paradigm for Academic Survey Automation18 Aug 2026
- 2026Multi-Perspective Agentic Program Repair via Code Property Graphs and Temporal Execution Graphs14 Jul 2026
- 2026Retrieval-Oriented Code Representations in Agentic Bug Localization13 Jul 2026
- 2026Writing Bug Reports for Software Repair Agents: What Information Matters Most?10 Jul 2026
- 2026Multi-Agent LLM Collaboration for Unit Test Generation via Human-Testing-Inspired Workflows10 Jul 2026
- 2026Beyond Textual Repository Exploration: Dual-Modal Structural Reasoning for Agentic Issue Resolution2 Jul 2026
- 2026ARISE: A Repository-level Graph Representation and Toolset for Agentic Program Repair and Fault Localization4 May 2026
- 2026LLM-Based Multi-Agent Systems for Code Generation: A Multi-Vocal Literature Review25 Feb 2026
- 2025HAFixAgent: History-Aware Program Repair Agent2 Nov 2025
- 2025AI Agentic Programming: A Survey of Techniques, Challenges, and Opportunities15 Aug 2025
- 2025Agentic Program Repair from Test Failures at Scale: A Neuro-symbolic approach with static analysis and test execution feedback24 Jul 2025
12 newest of 14 All 14 (JSON)
- 2024When LLMs Meet Cybersecurity: A Systematic Literature Reviewcited by 70
- 2025Type-Constrained Code Generation with Language Modelscited by 32
- 2024Source Code Summarization in the Era of Large Language Modelscited by 30
- 2024RepairBench: Leaderboard of Frontier Models for Program Repaircited by 9
- 2024Can GPT-O1 Kill All Bugs? An Evaluation of GPT-Family LLMs on QuixBugscited by 7
- 2024ESALE: Enhancing Code-Summary Alignment Learning for Source Code Summarizationcited by 6
- 2025Can LLMs Find Bugs in Code? An Evaluation from Beginner Errors to Security Vulnerabilities in Python and C++cited by 3
- 2026CktEvo: Repository-Level RTL Code Benchmark for Design Evolutioncited by 3
- 2024Automated Code Fix Suggestions for Accessibility Issues in Mobile Appscited by 2
- 2024An Empirical Study on Self-correcting Large Language Models for Data Science Code Generationcited by 2
- 2024Detection Made Easy: Potentials of Large Language Models for Solidity Vulnerabilitiescited by 2
- 2026Polaris: A G"odel Agent Framework for Small Language Models through Experience-Abstracted Policy Repaircited by 2
12 most cited of 18 All 18 (JSON)
- 2026The Vibe Shift in Software Engineering: Evaluating AI-Led Conversational Programming for Performance, Cognition, and Responsible Adoption9 Sep 2026
- 2026Benchmarking Large Language Models on Floating-Point Error Classification30 Jun 2026
- 2026DeepDiscovery: A Location-Inference Framework for Task-Level Repository Understanding22 Jun 2026
- 2026An Empirical Evaluation of Locally Deployed LLMs for Bug Detection in Python Code25 Apr 2026
- 2026A Metamorphic Testing Approach to Diagnosing Memorization in LLM-Based Program Repair23 Apr 2026
- 2026Polaris: A G"odel Agent Framework for Small Language Models through Experience-Abstracted Policy Repair24 Mar 2026
- 2026CktEvo: Repository-Level RTL Code Benchmark for Design Evolution10 Feb 2026
- 2025Learning Bug Context for PyTorch-to-JAX Translation with LLMs10 Oct 2025
- 2025Can LLMs Find Bugs in Code? An Evaluation from Beginner Errors to Security Vulnerabilities in Python and C++22 Aug 2025
- 2025Type-Constrained Code Generation with Language Models12 Apr 2025
- 2024RepairBench: Leaderboard of Frontier Models for Program Repair27 Sep 2024
- 2024Can GPT-O1 Kill All Bugs? An Evaluation of GPT-Family LLMs on QuixBugs16 Sep 2024
12 newest of 18 All 18 (JSON)