mlsanityData to 7 Oct 2026

Try Again, Don't Look Back: Blind Resampling Outperforms Self-Repair in Small Code Models

arXiv 2607.26117 ↗First posted 28 Jul 2026cs.SE cs.AI cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

Self-repair - returning a failed program to the model together with its test output and asking for a correction - is a standard component of code agents, and is almost always evaluated against a baseline that does not retry at all. We argue that this comparison confounds the value of the feedback with the value of the extra attempt. Using a placebo-controlled design on MBPP+ at three model scales (1.5B, 3B, 7B), we compare four matched-budget retry conditions: blind resampling, a content-free failure notice, genuine execution feedback, and feedback augmented with verbal self-reflection. Blind resampling is the strongest condition below 7B, and remains statistically tied with the best condition at 7B, while consuming 2.5-5.5x fewer tokens; conditioning on the model's own failed attempt costs 6.1 points at 1.5B (p=0.006), and the informational content of execution feedback adds nothing measurable over the placebo. We attribute this to anchoring: when shown its previous attempt, a model reproduces a near-identical program in 33-68% of retries, against 2-14% under blind resampling. Two further experiments delimit the effect. Retrieved solutions to other tasks change nothing (bounded to +/-3.5 points), which localizes the harm to self-conditioning rather than context length; and reflection, the only condition that measurably weakens the anchor, remains dominated on cost. Replication rules out two competing explanations: the penalty is unchanged at full precision, and it reproduces on an independent model family. Across six configurations spanning two families and two precisions, its magnitude is predicted by baseline quality alone (r=0.96) - the cost of anchoring is the cost of committing to a bad first attempt.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 12 of its 18 references that are in the map. First posted on arXiv 28 Jul 2026.

Builds on

12 of 18 references.

References · 12

  1. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  2. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  3. 2021Program Synthesis with Large Language Modelscited by 3,509
  4. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  5. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  6. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,628
  7. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,332
  8. 2024Qwen2.5-Coder Technical Reportcited by 1,151
  9. 2024Agentless: Demystifying LLM-based Software Engineering Agentscited by 400
  10. 2025Darwin Godel Machine: Open-Ended Evolution of Self-Improving Agentscited by 294
  11. 2023Is Self-Repair a Silver Bullet for Code Generation?cited by 191
  12. 2025SWE-Exp: Experience-Driven Software Issue Resolutioncited by 64

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.