mlsanityData to 14 Oct 2026

Dr. Boot: Bootstrapping Program Synthesis Language Models to Perform Repairing

arXiv 2507.15889 ↗First posted 20 Jul 2025cs.SE cs.AIcited by 0

On the map: Synthetic data and self-improvement for LLMs → Code and instruction data synthesis

Abstract (arXiv)

Language models for program synthesis are usually trained and evaluated on programming competition datasets (MBPP, APPS). However, these datasets are limited in size and quality, while these language models are extremely data hungry. Additionally, the language models have a misaligned program synthesis process compared to humans. While humans iteratively develop code with the help of a compiler, most program synthesis models currently produce code in one go. To solve these issues, we introduce a bootstrapping algorithm for program synthesis, that supports teaching models how to repair. We show that bootstrapping consistently outperforms regular fine-tuning. Compared to other work, our bootstrapped model performs on par with fine-tuned models that are 68% larger. Notably, bootstrapping with repairing also improves non-repairing performance compared to regular bootstrapping during inference. However, on our models, repairing during inference is likely inferior to simply sampling the same number of solutions. Furthermore, we find that there are issues with the example test cases in the training portion of the APPS dataset that are valuable to the community, as many repairing and reinforcement learning methods rely on them.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Code and instruction data synthesis, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 32 of its 42 references that are in the map. First posted on arXiv 20 Jul 2025.

Builds on

32 of 42 references.

References · 32

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  5. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  6. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  7. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  8. 2021Program Synthesis with Large Language Modelscited by 3,637
  9. 2023Teaching Large Language Models to Self-Debugcited by 988
  10. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 940
  11. 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 906
  12. 2021Measuring Coding Challenge Competence With APPScited by 902
20 more
  1. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 886
  2. 2023StarCoder: may the source be with you!cited by 799
  3. 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 733
  4. 2019Fast Transformer Decoding: One Write-Head is All You Needcited by 717
  5. 2023Textbooks Are All You Needcited by 569
  6. 2023Simple and Controllable Music Generationcited by 488
  7. 2022CodeT: Code Generation with Generated Testscited by 384
  8. 2022CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learningcited by 358
  9. 2023CodeT5+: Open Code Large Language Models for Code Understanding and Generationcited by 333
  10. 2023Is Self-Repair a Silver Bullet for Code Generation?cited by 197
  11. 2020DreamCoder: Growing generalizable, interpretable knowledge with wake-sleep Bayesian program learningcited by 161
  12. 2023CodeGen2: Lessons for Training LLMs on Programming and Natural Languagescited by 125
  13. 2023RLTF: Reinforcement Learning from Unit Test Feedbackcited by 114
  14. 2023Self-Edit: Fault-Aware Code Editor for Code Generationcited by 111
  15. 2023Execution-based Code Generation using Deep Reinforcement Learningcited by 99
  16. 2022Coder Reviewer Reranking for Code Generationcited by 84
  17. 2022Language Models Can Teach Themselves to Program Bettercited by 78
  18. 2023Improving Code Generation by Training with Natural Language Feedbackcited by 76
  19. 2023Reprompting: Automated Chain-of-Thought Prompt Inference Through Gibbs Samplingcited by 47
  20. 2021Programming Puzzlescited by 25

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Bootstrapping Code Translation with Weighted Multilanguage Exploration
  2. 2025Self-Bootstrapping Automated Program Repair: Using LLMs to Generate and Evaluate Synthetic Training Data for Bug Repair
  3. 2024RePair: Automated Program Repair with Process-based Feedback
  4. 2026CodePivot: Bootstrapping Multilingual Transpilation in LLMs via Reinforcement Learning without Parallel Corpora
  5. 2026BoostAPR: Boosting Automated Program Repair via Execution-Grounded Reinforcement Learning with Dual Reward Models
  6. 2026Learn2Zinc: Fine-tuning Small Language Models for Text-to-Model Translation in MiniZinc
  7. 2024Code Repair with LLMs gives an Exploration-Exploitation Tradeoff
  8. 2025Repair-R1: Better Test Before Repair