# From Empirical Evaluation to Context-Aware Enhancement: Repairing Regression Errors with LLMs

arXiv: https://arxiv.org/abs/2506.13182 · HTML: https://mlsanity.com/paper/2506.13182/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 62 references that are in the map.
- First posted on arXiv 16 Jun 2025.

## Builds on (most cited first, 28 of 28)

- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023)
- [CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis](https://mlsanity.com/paper/2203.13474/) (2022)
- [InCoder: A Generative Model for Code Infilling and Synthesis](https://mlsanity.com/paper/2204.05999/) (2022)
- [RepairAgent: An Autonomous, LLM-Based Agent for Program Repair](https://mlsanity.com/paper/2403.17134/) (2024)
- [Keep the Conversation Going: Fixing 162 out of 337 bugs for 0.42 each using ChatGPT](https://mlsanity.com/paper/2304.00385/) (2023)
- [CURE: Code-Aware Neural Machine Translation for Automatic Program Repair](https://mlsanity.com/paper/2103.00073/) (2021)
- [Refining ChatGPT-Generated Code: Characterizing and Mitigating Code Quality Issues](https://mlsanity.com/paper/2307.12596/) (2023)
- [SpecRover: Code Intent Extraction via LLMs](https://mlsanity.com/paper/2408.02232/) (2024)
- [TBar: Revisiting Template-based Automated Program Repair](https://mlsanity.com/paper/1903.08409/) (2019)
- [A Syntax-Guided Edit Decoder for Neural Program Repair](https://mlsanity.com/paper/2106.08253/) (2021)
- [Neural Program Repair with Execution-based Backpropagation](https://mlsanity.com/paper/2105.04123/) (2021)
- [RepairLLaMA: Efficient Representations and Fine-Tuned Adapters for Program Repair](https://mlsanity.com/paper/2312.15698/) (2023)
- [BugsInPy: A Database of Existing Bugs in Python Programs to Enable Controlled Testing and Debugging Studies](https://mlsanity.com/paper/2401.15481/) (2024)
- [Nopol: Automatic Repair of Conditional Statement Bugs in Java Programs](https://mlsanity.com/paper/1811.04211/) (2018)
- [Enhancing repository-level software repair via repository-aware knowledge graphs](https://mlsanity.com/paper/2503.21710/) (2025)
- [ARJA: Automated Repair of Java Programs via Multi-Objective Genetic Programming](https://mlsanity.com/paper/1712.07804/) (2017)
- [Hybrid Automated Program Repair by Combining Large Language Models and Program Analysis](https://mlsanity.com/paper/2406.00992/) (2024)
- [How Far Can We Go with Practical Function-Level Program Repair?](https://mlsanity.com/paper/2404.12833/) (2024)
- [Dissection of a Bug Dataset: Anatomy of 395 Patches from Defects4J](https://mlsanity.com/paper/1801.06393/) (2018)
- [Bears: An Extensible Java Bug Benchmark for Automatic Program Repair Studies](https://mlsanity.com/paper/1901.06024/) (2019)
- [BugSwarm: Mining and Continuously Growing a Dataset of Reproducible Failures and Fixes](https://mlsanity.com/paper/1903.06725/) (2019)
- [Aligning the Objective of LLM-based Program Repair](https://mlsanity.com/paper/2404.08877/) (2024)
- [When Fine-Tuning LLMs Meets Data Privacy: An Empirical Study of Federated Learning in LLM-Based Program Repair](https://mlsanity.com/paper/2412.01072/) (2024)
- [Ultra-Large Repair Search Space with Automatically Mined Templates: the Cardumen Mode of Astor](https://mlsanity.com/paper/1712.03854/) (2017)
- [Fonte: Finding Bug Inducing Commits from Failures](https://mlsanity.com/paper/2212.06376/) (2022)
- [RegMiner: Towards Constructing a Large Regression Dataset from Code Evolution History](https://mlsanity.com/paper/2109.12389/) (2021)
- [Critical Review of BugSwarm for Fault Localization and Program Repair](https://mlsanity.com/paper/1905.09375/) (2019)

## Built on by (most cited first, 1 of 1)

- [Enhancing Neural Code Representation with Additional Context](https://mlsanity.com/paper/2510.12082/) (2025)

## Abstract (arXiv)

[...] Since then, various APR approaches, especially those leveraging the power of large language models (LLMs), have been rapidly developed to fix general software bugs. Unfortunately, the effectiveness of these advanced techniques in the context of regression bugs remains largely unexplored. This gap motivates the need for an empirical study evaluating the effectiveness of modern APR techniques in fixing real-world regression bugs. In this work, we conduct an empirical study of APR techniques on regression bugs. To facilitate our study, we introduce RegressionBug4APR, a high-quality benchmark of Java and Python regression bugs integrated into a framework designed to facilitate APR research. The current benchmark includes 200 regression bugs collected from widely used real-world GitHub repositories. We begin by conducting an in-depth analysis of the benchmark, demonstrating its diversity and quality. Building on this foundation, we empirically evaluate the capabilities of APR to regression bugs by assessing both traditional APR tools and advanced LLM-based APR approaches. Our experimental results show that classical APR tools fail to repair any bugs, while LLM-based APR approaches exhibit promising potential. Motivated by these results, we investigate impact of incorporating bug-inducing change information into LLM-based APR approaches for fixing regression bugs. We further conduct an ablation study to disaggregate the contribution of each contextual element within the bug-inducing change information. Our results highlight that this context-aware enhancement significantly improves the performance of LLM-based APR, yielding 1.6x more successful repairs compared to using LLM-based APR without such context. Moreover, our findings are consistent across both Java and Python benchmarks, providing preliminary evidence for the generalizability of our findings.