# GAMMA: Revisiting Template-based Automated Program Repair via Mask Prediction

arXiv: https://arxiv.org/abs/2309.09308 · HTML: https://mlsanity.com/paper/2309.09308/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Sep 2023.

## Built on by (most cited first, 23 of 23)

- [From Code to Correctness: Closing the Last Mile of Code Generation with Hierarchical Debugging](https://mlsanity.com/paper/2410.01215/) (2024)
- [How secure is AI-generated Code: A Large-Scale Comparison of Large Language Models](https://mlsanity.com/paper/2404.18353/) (2024)
- [RePair: Automated Program Repair with Process-based Feedback](https://mlsanity.com/paper/2408.11296/) (2024)
- [CigaR: Cost-efficient Program Repair with LLMs](https://mlsanity.com/paper/2402.06598/) (2024)
- [SGAgent: Suggestion-Guided LLM-Based Multi-Agent Framework for Repository-Level Software Repair](https://mlsanity.com/paper/2602.23647/) (2026)
- [Can GPT-O1 Kill All Bugs? An Evaluation of GPT-Family LLMs on QuixBugs](https://mlsanity.com/paper/2409.10033/) (2024)
- [TransAgent: Enhancing LLM-Based Code Translation via Fine-Grained Execution Alignment](https://mlsanity.com/paper/2409.19894/) (2024)
- [Repair-R1: Better Test Before Repair](https://mlsanity.com/paper/2507.22853/) (2025)
- [Memory-Efficient Large Language Models for Program Repair with Semantic-Guided Patch Generation](https://mlsanity.com/paper/2410.16655/) (2024)
- [Integrating Various Software Artifacts for Better LLM-based Bug Localization and Program Repair](https://mlsanity.com/paper/2412.03905/) (2024)
- [Improving Deep Assertion Generation via Fine-Tuning Retrieval-Augmented Pre-trained Language Models](https://mlsanity.com/paper/2502.16071/) (2025)
- [Evaluating the Generalizability of LLMs in Automated Program Repair](https://mlsanity.com/paper/2503.09217/) (2025)
- [DynaFix: Iterative Automated Program Repair Driven by Execution-Level Dynamic Information](https://mlsanity.com/paper/2512.24635/) (2025)
- [Improving Retrieval-Augmented Deep Assertion Generation via Joint Training](https://mlsanity.com/paper/2502.10696/) (2025)
- [FailureMem: A Failure-Aware Multimodal Framework for Autonomous Software Repair](https://mlsanity.com/paper/2603.17826/) (2026)
- [Accelerating Automatic Program Repair with Dual Retrieval-Augmented Fine-Tuning and Patch Generation on Large Language Models](https://mlsanity.com/paper/2507.10103/) (2025)
- [Empirical Study of Code Large Language Models for Binary Security Patch Detection](https://mlsanity.com/paper/2509.06052/) (2025)
- [ReProAgent: Tool-Augmented Multi-Stage Agentic Generation of Bug Reproduction Tests from Issue Reports](https://mlsanity.com/paper/2607.09123/) (2026)
- [Multi-Perspective Agentic Program Repair via Code Property Graphs and Temporal Execution Graphs](https://mlsanity.com/paper/2607.12605/) (2026)
- [T3: Multi-level Tree-based Automatic Program Repair with Large Language Models](https://mlsanity.com/paper/2506.21211/) (2025)
- [Securing Code Understanding: Detecting Natural Backdoor Vulnerability in Code Language Models](https://mlsanity.com/paper/2606.10846/) (2026)
- [Multi-Agent LLM Collaboration for Unit Test Generation via Human-Testing-Inspired Workflows](https://mlsanity.com/paper/2607.09101/) (2026)
- [MultiFixer: A Coordinator-Proposer Based Multi-Agent Framework For Fixing Multi-Hunk Bugs](https://mlsanity.com/paper/2607.26591/) (2026)

## Abstract (arXiv)

Automated program repair (APR) aims to fix software bugs without human intervention and template-based APR has been widely investigated with promising results. However, it is challenging for template-based APR to select the appropriate donor code, which is an important repair ingredient for generating candidate patches. Inappropriate donor code may cause plausible but incorrect patch generation even with correct fix patterns, limiting the repair performance. In this paper, we aim to revisit template-based APR, and propose GAMMA, to directly leverage large pre-trained language models for donor code generation. Our main insight is that instead of retrieving donor code in the local buggy file, we can directly predict the correct code tokens based on the context code snippets and repair patterns by a cloze task. Specifically, (1) GAMMA revises a variety of fix templates from state-of-the-art template-based APR techniques (i.e., TBar) and transforms them into mask patterns. (2) GAMMA adopts a pre-trained language model to predict the correct code for masked code as a fill-in-the-blank task. The experimental results demonstrate that GAMMA correctly repairs 82 bugs on Defects4J-v1.2, which achieves 20.59% (14 bugs) and 26.15% (17 bugs) improvement over the previous state-of-the-art template-based approach TBar and learning-based one Recoder. Furthermore, GAMMA repairs 45 bugs and 22 bugs from the additional Defects4J-v2.0 and QuixBugs, indicating the generalizability of GAMMA in addressing the dataset overfitting issue. We also prove that adopting other pre-trained language models can provide substantial advancement, e.g., CodeBERT-based and ChatGPT-based GAMMA is able to fix 80 and 67 bugs on Defects4J-v1.2, indicating the scalability of GAMMA. Overall, our study highlights the promising future of adopting pre-trained models to generate correct patches on top of fix patterns.