mlsanityData to 14 Oct 2026

Code Repair with LLMs gives an Exploration-Exploitation Tradeoff

arXiv 2405.17503 ↗First posted 26 May 2024cs.SE cs.AI cs.CLcited by 51

Not on the map. Why?

Abstract (arXiv)

Iteratively improving and repairing source code with large language models (LLMs), known as refinement, has emerged as a popular way of generating programs that would be too complex to construct in one shot. Given a bank of test cases, together with a candidate program, an LLM can improve that program by being prompted with failed test cases. But it remains an open question how to best iteratively refine code, with prior work employing simple greedy or breadth-first strategies. We show here that refinement exposes an explore-exploit tradeoff: exploit by refining the program that passes the most test cases, or explore by refining a lesser considered program. We frame this as an arm-acquiring bandit problem, which we solve with Thompson Sampling. The resulting LLM-based program synthesis algorithm is broadly applicable: Across loop invariant synthesis, visual reasoning puzzles, and competition programming problems, we find that our new method can solve more problems using fewer language model calls.

Key facts. Cited by 51 papers in the map (data through 14 Oct 2026). Builds on 22 of its 41 references that are in the map. First posted on arXiv 26 May 2024.

Builds on

22 of 41 references.

References · 22

  1. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  2. 2021Program Synthesis with Large Language Modelscited by 3,637
  3. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  4. 2022Competition-Level Code Generation with AlphaCodecited by 1,462
  5. 2023Teaching Large Language Models to Self-Debugcited by 988
  6. 2021Measuring Coding Challenge Competence With APPScited by 902
  7. 2017A Tutorial on Thompson Samplingcited by 433
  8. 2022CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learningcited by 358
  9. 2022Generating Sequences by Learning to Self-Correctcited by 277
  10. 2023Is Self-Repair a Silver Bullet for Code Generation?cited by 197
  11. 2022Practical Program Repair in the Era of Large Pre-trained Language Modelscited by 133
  12. 2023Hypothesis Search: Inductive Reasoning with Language Modelscited by 120
10 more
  1. 2024WorldCoder, a Model-Based LLM Agent: Building World Models by Writing Code and Interacting with the Environmentcited by 107
  2. 2023Phenomenal Yet Puzzling: Testing Inductive Reasoning Capabilities of Language Models with Hypothesis Refinementcited by 89
  3. 2022Automated Repair of Programs from Large Language Modelscited by 71
  4. 2022Parsel: Algorithmic Reasoning with Language Models by Composing Decompositionscited by 64
  5. 2024CYCLE: Learning to Self-Refine the Code Generationcited by 42
  6. 2023WhiteFox: White-Box Compiler Fuzzing Empowered by Large Language Modelscited by 37
  7. 2023Finding Inductive Loop Invariants using Large Language Modelscited by 36
  8. 2021Fast and flexible: Human program induction in abstract reasoning taskscited by 33
  9. 2020Learning Nonlinear Loop Invariants with Gated Continuous Logic Networks (Extended Version)cited by 17
  10. 2020Global Guidance for Local Generalization in Model Checkingcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 9; 2025: 18; 2026: 24.

51 papers.

Sort:

By area:

  1. 2026AutoHarness: improving LLM agents by automatically synthesizing a code harnesscited by 76
  2. 2025ReasonFlux: Hierarchical LLM Reasoning via Scaling Thought Templatescited by 63
  3. 2024Combining Induction and Transduction for Abstract Reasoningcited by 53
  4. 2025Wider or Deeper? Scaling LLM Inference-Time Compute with Adaptive Branching Tree Searchcited by 51
  5. 2025Self-Improving Language Models for Evolutionary Program Synthesis: A Case Study on ARC-AGIcited by 36
  6. 2025CodeJudgeBench: Benchmarking LLM-as-a-Judge for Coding Taskscited by 33
  7. 2024What Makes Large Language Models Reason in (Multi-Turn) Code Generation?cited by 29
  8. 2024Is Programming by Example solved by LLMs?cited by 27
  9. 2025PoE-World: Compositional World Modeling with Products of Programmatic Expertscited by 26
  10. 2024SELA: Tree-Search Enhanced LLM Agents for Automated Machine Learningcited by 24
  11. 2024Reasoning Through Execution: Unifying Process and Outcome Rewards for Code Generationcited by 21
  12. 2026Calibrate-Then-Act: Cost-Aware Exploration in LLM Agentscited by 17
38 more
  1. 2026LoopViT: Scaling Visual ARC with Looped Transformerscited by 15
  2. 2026M2F: Automated Formalization of Mathematical Literature at Scalecited by 15
  3. 2025LLM-Guided Probabilistic Program Induction for POMDP Model Estimationcited by 12
  4. 2024Scattered Forest Search: Smarter Code Space Exploration with LLMscited by 10
  5. 2025Token Hidden Reward: Steering Exploration-Exploitation in Group Relative Deep Reinforcement Learningcited by 10
  6. 2025ARC-GEN: A Mimetic Procedural Benchmark Generator for the Abstraction and Reasoning Corpuscited by 10
  7. 2025Improved Generalized Planning with LLMs through Strategy Refinement and Reflectioncited by 7
  8. 2025InfCode: Adversarial Iterative Refinement of Tests and Patches for Reliable Software Issue Resolutioncited by 7
  9. 2024Relational decomposition for program synthesiscited by 6
  10. 2024Generating Diverse Hypotheses for Inductive Reasoningcited by 6
  11. 2025Multi-Agent Code-Orchestrated Generation for Reliable Infrastructure-as-Codecited by 5
  12. 2026How Many Tries Does It Take? Iterative Self-Repair in LLM Code Generation Across Model Scales and Benchmarkscited by 5
  13. 2026Distilling Game Code World Model Generation into Lightweight Large Language Modelscited by 5
  14. 2025Program Synthesis via Test-Time Transductioncited by 4
  15. 2026Automating Computational Reproducibility in Social Science: Comparing Prompt-Based and Agent-Based Approachescited by 4
  16. 2026Compute Allocation for Self-Evolving LLMs: From Depth-Breadth to Multi-Armed Banditscited by 4
  17. 2026Twin: Playing an Unknown Game with a Test-Time Digital Twincited by 4
  18. 2025Cog-Rethinker: Hierarchical Metacognitive Reinforcement Learning for LLM Reasoningcited by 3
  19. 2026Exploration and Exploitation Errors Are Measurable for Language Model Agentscited by 3
  20. 2025Sample-Efficient Behavior Cloning Using General Domain Knowledgecited by 2
  21. 2026More Bang for the Buck: Improving the Inference of Large Language Models at a Fixed Budget using Reset and Discard (ReD)cited by 2
  22. 2026TriVAL: A Tri-Validation Framework for Faithful Automatic Optimization Modelingcited by 2
  23. 2025Collaborative Agents for Automated Program Repair in Rubycited by 1
  24. 2026A Component-Based Survey of Interactions between Large Language Models and Multi-Armed Banditscited by 1
  25. 2026Auto Researching, not hyperparameter tuning: Convergence Analysis of 10,000 Experimentscited by 1
  26. 2026Obey, Diverge, Collapse: Blind Obedience to Incorrect Instructions Drives Code LLMs to Irrecoverable Code Semantic Collapsecited by 1
  27. 2026SkillHEX: Improving Agent Skills via Hypothesis-Driven Autonomous Exploration and Exploitationcited by 1
  28. 2026Two-Stage Reinforcement Learning for Sound and Adversarial Test Generation in Code LLMscited by 1
  29. 2024System 2 Reasoning for Human-AI Alignment: Generality and Adaptivity via ARC-AGIcited by 0
  30. 2025RN-F: A Novel Approach for Mitigating Contaminated Data in Large Language Modelscited by 0
  31. 2025TGPR: Tree-Guided Policy Refinement for Robust Self-Debugging of LLMscited by 0
  32. 2025What a diff makes: automating code migration with large language modelscited by 0
  33. 2026The Critical Horizon: Inspection Design Principles for Multi-Stage Operations and Deep Reasoningcited by 0
  34. 2026SemRep: Generative Code Representation Learning with Code Transformationscited by 0
  35. 2026Progressive Content Refinement with Decaying Reward Joint LinUCBcited by 0
  36. 2026Discovering Adaptive Transmission Programs for Collective Innovationcited by 0
  37. 2026Test-Driven Approaches to Software Engineering with Large Language Models: A Survey of Phases, Tasks, and Agent Skillscited by 0
  38. 2026Generate to Explore, Select to Exploit: Aligning LLM-based Headline Generation with Personalized Recommendationcited by 0

50 most cited of 51 All 51 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Unlocking LLM Repair Capabilities Through Cross-Language Translation and Multi-Agent Refinement
  2. 2026Unlocking LLM Code Correction with Iterative Feedback Loops
  3. 2025Fully Autonomous Programming using Iterative Multi-Agent Debugging with Large Language Models
  4. 2025Let's Revise Step-by-Step: A Unified Local Search Framework for Code Generation with LLMs
  5. 2026If It's Not Buggy, Don't Fix It: On the Dynamics of Iterative Bug-fixing with LLMs
  6. 2024Rethinking Code Refinement: Learning to Judge Code Efficiency
  7. 2025The Art of Repair: Optimizing Iterative Program Repair with Instruction-Tuned Models
  8. 2026Refine After Generation: Toward Correct and Concise Patches in LLM-based Program Repair