mlsanityData to 14 Oct 2026

Self-Bootstrapping Automated Program Repair: Using LLMs to Generate and Evaluate Synthetic Training Data for Bug Repair

arXiv 2505.07372 ↗First posted 12 May 2025cs.SE cs.AIcited by 0

On the map: Synthetic data and self-improvement for LLMs → Code and instruction data synthesis

Abstract (arXiv)

This paper presents a novel methodology for enhancing Automated Program Repair (APR) through synthetic data generation utilizing Large Language Models (LLMs). Current APR systems are constrained by the limited availability of high-quality training data encompassing diverse bug types across multiple programming languages. The proposed approach addresses this limitation through a two-phase process: a synthetic sample generation followed by a rigorous quality assessment. Multiple state-of-the-art LLMs were employed to generate approximately 30,000 paired examples of buggy and fixed code across 12 programming languages and 13 bug categories. Subsequently, these samples underwent cross-model evaluation against five criteria: correctness, code quality, security, performance, and completeness. Experimental evaluation on the VulRepair test set dataset showed statistically significant improvements in Perfect Prediction rates, with the quality-filtered synthetic dataset achieving 17.18% (Top@1) and 23.00% (Top@5) compared to the baseline's 11.68% and 18.88% respectively, representing a 47% relative improvement in Top@1 and 22% in Top@5. The methodology was validated through rigorous statistical testing, including ANOVA and post-hoc Tukey's Honest Significant Difference analysis. Furthermore, the best-performing configurations surpassed existing systems despite using a less computationally intensive decoding strategy. This research establishes a self-bootstrapping paradigm in which LLMs generate and evaluate their own training data, suggesting promising directions for addressing data scarcity in similar software engineering tasks and advancing the development of robust, adaptable tools for automated code maintenance.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Code and instruction data synthesis, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 24 of its 47 references that are in the map. First posted on arXiv 12 May 2025.

Builds on

24 of 47 references.

References · 24

  1. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  2. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  3. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,310
  4. 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 906
  5. 2018SequenceR: Sequence-to-Sequence Learning for End-to-End Program Repaircited by 145
  6. 2022Practical Program Repair in the Era of Large Pre-trained Language Modelscited by 133
  7. 2023InferFix: End-to-End Program Repair with LLMscited by 114
  8. 2018An Empirical Study on Learning Bug-Fixing Patches in the Wild via Neural Machine Translationcited by 108
  9. 2023RTLFixer: Automatically Fixing RTL Syntax Errors with Large Language Modelscited by 100
  10. 2021CVEfixes: Automated Collection of Vulnerabilities and Their Fixes from Open-Source Softwarecited by 91
  11. 2022Less Training, More Repairing Please: Revisiting Automated Program Repair via Zero-shot Learningcited by 76
  12. 2019TBar: Revisiting Template-based Automated Program Repaircited by 65
12 more
  1. 2023A Survey of Learning-based Automated Program Repaircited by 55
  2. 2023RAP-Gen: Retrieval-Augmented Patch Generation with CodeT5 for Automatic Program Repaircited by 44
  3. 2022SelfAPR: Self-supervised Program Repair with Test Execution Diagnosticscited by 33
  4. 2024Evaluating Language Models as Synthetic Data Generatorscited by 33
  5. 2018Nopol: Automatic Repair of Conditional Statement Bugs in Java Programscited by 32
  6. 2022Execution-based Evaluation for Data Science Code Generation Modelscited by 32
  7. 2017ARJA: Automated Repair of Java Programs via Multi-Objective Genetic Programmingcited by 26
  8. 2018FixMiner: Mining Relevant Fix Patterns for Automated Program Repaircited by 22
  9. 2024ContrastRepair: Enhancing Conversation-Based Automated Program Repair via Contrastive Test Case Pairscited by 19
  10. 2024Enhanced Automated Code Vulnerability Repair using Large Language Modelscited by 10
  11. 2017Ultra-Large Repair Search Space with Automatically Mined Templates: the Cardumen Mode of Astorcited by 9
  12. 2024Exploring Data-Efficient Adaptation of Large Language Models for Code Generationcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Unlocking LLM Repair Capabilities Through Cross-Language Translation and Multi-Agent Refinement
  2. 2025The Art of Repair: Optimizing Iterative Program Repair with Instruction-Tuned Models
  3. 2025Accelerating Automatic Program Repair with Dual Retrieval-Augmented Fine-Tuning and Patch Generation on Large Language Models
  4. 2025SWE-Synth: Synthesizing Verifiable Bug-Fix Data to Enable Large Language Models in Resolving Real-World Bugs
  5. 2026Large Language Models for Programming: Actually Fixing or Reimplementing Incorrect Code?
  6. 2025Repair-R1: Better Test Before Repair
  7. 2026If It's Not Buggy, Don't Fix It: On the Dynamics of Iterative Bug-fixing with LLMs
  8. 2024Aligning the Objective of LLM-based Program Repair