Data to 14 Oct 2026

Falsification, Not Exposure: An Internally Preregistered Placebo-Controlled Decomposition of Self-Repair Feedback in Frozen Small Code Models

arXiv 2606.31511 ↗First posted 30 Jun 2026cs.SE cs.CL cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

In deployment settings where retraining is infeasible, small frozen code models are routinely asked to repair a failed program after seeing their own failing output, usually treated as a retry mechanism. From a Popperian view, a generated program is a conjecture and a test-execution violation is an oracle-relative, executable counterexample, so feedback's value should be attributed not to re-exposure to failing code but to whether the conjecture is opened to external, executable criticism. As the third stage of a falsification-centered measurement program, this study builds a placebo-controlled instrument that decomposes the feedback packet against a blind-resampling baseline at matched output-generation budget and against content-free, shape-matched placebos. The contribution is not a new repair algorithm but a reflexive methodology (packet decomposition, placebo mirroring, matched-budget discordant-pair tests, fresh-generation confirmation, executable audits) that makes both the model's program conjecture and the researcher's "feedback content works" claim falsifiable. Across six HumanEval+/MBPP+ cells with three 0.5B-1.5B frozen models, 290 dead task-cell units (no best-of-8 candidate passing the public tier) were evaluated; the main run produced 7,000 fresh generations and a preregistered follow-up 1,400 more. Blind resampling exceeded bare-code retry by +18 net unlocks (25/7, Holm p=0.0021). Code-plus-facts recovered +18 over bare code (21/3, p=0.00042) and +15 over a generic-bullet placebo (p=0.0041). An instruction-only effect was not distinguishable (+3, p=0.36). Code-plus-facts and blind resampling tied at 26 unlocks each (not equivalence). Six external-controller follow-ups tied a content-free shape placebo. In this regime, falsification helped not as vocabulary or self-critique, but as comparison with external, executable counterexamples.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 39 of its 53 references that are in the map. First posted on arXiv 30 Jun 2026.

Builds on

39 of 53 references.

References · 39

2021Evaluating Large Language Models Trained on Codecited by 8,320
2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
2021Program Synthesis with Large Language Modelscited by 3,637
2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,986
2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,391
2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,181
2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 1,017
2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 991
2023Teaching Large Language Models to Self-Debugcited by 988
2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 817
2024LLM Evaluators Recognize and Favor Their Own Generationscited by 686

27 more

2024OpenCodeInterpreter: Integrating Code Generation with Execution and Refinementcited by 247
2023Is Self-Repair a Silver Bullet for Code Generation?cited by 197
2023Software Testing with Large Language Models: Survey, Landscape, and Visioncited by 166
2024On the Self-Verification Limitations of Large Language Models on Reasoning and Planning Taskscited by 142
2024Debug like a Human: A Large Language Model Debugger via Verifying Runtime Execution Step-by-stepcited by 127
2023Can Large Language Models Really Improve by Self-critiquing Their Own Plans?cited by 108
2022Capturing Failures of Large Language Models via Human Cognitive Biasescited by 98
2024Inference-Aware Fine-Tuning for Best-of-N Sampling in Large Language Modelscited by 89
2024Planning In Natural Language Improves LLM Search For Code Generationcited by 83
2024DebugBench: Evaluating Debugging Capability of Large Language Modelscited by 76
2024Learning How Hard to Think: Input-Adaptive Allocation of LM Computationcited by 70
2023Towards an Understanding of Large Language Models in Software Engineering Taskscited by 68
2024CYCLE: Learning to Self-Refine the Code Generationcited by 42
2025When To Solve, When To Verify: Compute-Optimal Problem Solving and Generative Verification for LLM Reasoningcited by 42
2023ALGO: Synthesizing Algorithmic Programs with LLM-Generated Oracle Verifierscited by 41
2024LeDex: Training LLMs to Better Self-Debug and Explain Codecited by 40
2023Large Language Models of Code Fail at Completing Code with Potential Bugscited by 29
2024What Makes Large Language Models Reason in (Multi-Turn) Code Generation?cited by 29
2023Chain-of-Thought in Neural Code Generation: From and For Lightweight Language Modelscited by 27
2024Divide-and-Conquer Meets Consensus: Unleashing the Power of Functions in Code Generationcited by 19
2023Pre-registration for Predictive Modelingcited by 13
2024The Impact of Prompt Programming on Function-Level Code Generationcited by 13
2025Execution Guided Line-by-Line Code Generationcited by 10
2025Uncovering Systematic Failures of LLMs in Verifying Code Against Natural Language Specificationscited by 5
2025Self-Correcting Code Generation Using Small Language Modelscited by 4
2026Scaffold, Not Vocabulary? A Controlled, Two-Tier, Pre-Registered Study of a Popperian Code-Generation Skillcited by 3
2026Selection Without Signal, Recovery Through Expression: A Measurement Study of Post-Hoc Falsification Operators for Frozen Small Code Modelscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.