# A Systematic Literature Review on Large Language Models for Automated Program Repair

arXiv: https://arxiv.org/abs/2405.01466 · HTML: https://mlsanity.com/paper/2405.01466/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 May 2024.

## Built on by (most cited first, 30 of 37)

- [Agentless: Demystifying LLM-based Software Engineering Agents](https://mlsanity.com/paper/2407.01489/) (2024)
- [When LLMs Meet Cybersecurity: A Systematic Literature Review](https://mlsanity.com/paper/2405.03644/) (2024)
- [Type-Constrained Code Generation with Language Models](https://mlsanity.com/paper/2504.09246/) (2025)
- [Source Code Summarization in the Era of Large Language Models](https://mlsanity.com/paper/2407.07959/) (2024)
- [AI Agentic Programming: A Survey of Techniques, Challenges, and Opportunities](https://mlsanity.com/paper/2508.11126/) (2025)
- [debug-gym: A Text-Based Environment for Interactive Debugging](https://mlsanity.com/paper/2503.21557/) (2025)
- [Afterburner: Reinforcement Learning Facilitates Self-Improving Code Efficiency Optimization](https://mlsanity.com/paper/2505.23387/) (2025)
- [Seeing is Fixing: Cross-Modal Reasoning with Multimodal LLMs for Visual Software Issue Fixing](https://mlsanity.com/paper/2506.16136/) (2025)
- [RepairBench: Leaderboard of Frontier Models for Program Repair](https://mlsanity.com/paper/2409.18952/) (2024)
- [ESALE: Enhancing Code-Summary Alignment Learning for Source Code Summarization](https://mlsanity.com/paper/2407.01646/) (2024)
- [Can GPT-O1 Kill All Bugs? An Evaluation of GPT-Family LLMs on QuixBugs](https://mlsanity.com/paper/2409.10033/) (2024)
- [Agentic Program Repair from Test Failures at Scale: A Neuro-symbolic approach with static analysis and test execution feedback](https://mlsanity.com/paper/2507.18755/) (2025)
- [Can LLMs Find Bugs in Code? An Evaluation from Beginner Errors to Security Vulnerabilities in Python and C++](https://mlsanity.com/paper/2508.16419/) (2025)
- [HAFixAgent: History-Aware Program Repair Agent](https://mlsanity.com/paper/2511.01047/) (2025)
- [CktEvo: Repository-Level RTL Code Benchmark for Design Evolution](https://mlsanity.com/paper/2603.08718/) (2026)
- [LLM-Based Multi-Agent Systems for Code Generation: A Multi-Vocal Literature Review](https://mlsanity.com/paper/2604.16321/) (2026)
- [Automated Code Fix Suggestions for Accessibility Issues in Mobile Apps](https://mlsanity.com/paper/2408.03827/) (2024)
- [An Empirical Study on Self-correcting Large Language Models for Data Science Code Generation](https://mlsanity.com/paper/2408.15658/) (2024)
- [Detection Made Easy: Potentials of Large Language Models for Solidity Vulnerabilities](https://mlsanity.com/paper/2409.10574/) (2024)
- [Improving Retrieval-Augmented Deep Assertion Generation via Joint Training](https://mlsanity.com/paper/2502.10696/) (2025)
- [Polaris: A G"odel Agent Framework for Small Language Models through Experience-Abstracted Policy Repair](https://mlsanity.com/paper/2603.23129/) (2026)
- [ARISE: A Repository-level Graph Representation and Toolset for Agentic Program Repair and Fault Localization](https://mlsanity.com/paper/2605.03117/) (2026)
- [DeepDiscovery: A Location-Inference Framework for Task-Level Repository Understanding](https://mlsanity.com/paper/2606.22906/) (2026)
- [Writing Bug Reports for Software Repair Agents: What Information Matters Most?](https://mlsanity.com/paper/2607.09553/) (2026)
- [Multi-Perspective Agentic Program Repair via Code Property Graphs and Temporal Execution Graphs](https://mlsanity.com/paper/2607.12605/) (2026)
- [T3: Multi-level Tree-based Automatic Program Repair with Large Language Models](https://mlsanity.com/paper/2506.21211/) (2025)
- [Learning Bug Context for PyTorch-to-JAX Translation with LLMs](https://mlsanity.com/paper/2510.09898/) (2025)
- [A Metamorphic Testing Approach to Diagnosing Memorization in LLM-Based Program Repair](https://mlsanity.com/paper/2604.21579/) (2026)
- [An Empirical Evaluation of Locally Deployed LLMs for Bug Detection in Python Code](https://mlsanity.com/paper/2604.23361/) (2026)
- [BLAgent: Agentic RAG for File-Level Bug Localization](https://mlsanity.com/paper/2605.17965/) (2026)

## Abstract (arXiv)

Automated Program Repair (APR) attempts to patch software bugs and reduce manual debugging efforts. Very recently, with the advances in Large Language Models (LLMs), an increasing number of APR techniques have been proposed, facilitating software development and maintenance and demonstrating remarkable performance. However, due to ongoing explorations in the LLM-based APR field, it is challenging for researchers to understand the current achievements, challenges, and potential opportunities. This work provides the first systematic literature review to summarize the applications of LLMs in APR between 2020 and 2025. We analyze 189 relevant papers from LLMs, APR and their integration perspectives. First, we categorize existing popular LLMs that are applied to support APR and outline four types of utilization strategies for their deployment. Besides, we detail some specific repair scenarios that benefit from LLMs, e.g., semantic bugs and security vulnerabilities. Furthermore, we discuss several critical aspects of integrating LLMs into APR research, e.g., input forms and open science. Finally, we highlight a set of challenges remaining to be investigated and the potential guidelines for future research. Overall, our paper provides a systematic overview of the research landscape to the APR community, helping researchers gain a comprehensive understanding of achievements and promote future research.