When AI Reviews Its Own Code: Recursive Self-Training Collapse in Code LLMs
On the map: Synthetic data and self-improvement for LLMs → Code and instruction data synthesis
Abstract (arXiv)
Recursive self-training can degrade neural generative models when generated data is reused without fresh human data or external quality control. We study this risk in code LLMs, where AI-generated code can enter real repositories, later become training data, and create a repository-scale self-training loop. While software development traditionally interrupts this loop through pull-request review, tests, compilation, and human approval, AI coding tools now produce code faster than humans can review it, and code review itself is increasingly automated by AI systems. We therefore compare three recursive fine-tuning regimes: no review, human-style exogenous-gate review using model-independent filters such as compilation and static quality checks, and AI-self-gate review using the code LLM's own signals such as perplexity and binary self-scoring. Across multiple code LLMs and benchmarks, every tested recursive regime falls below its own starting point; exogenous filters preserve executable validity and sometimes reduce decline, while AI-self-gate filters can look strong early but later lose their filtering effect. In the clearest case, the binary self-gate enters a rubber-stamp regime where acceptance scores rise while benchmark correctness falls. We explain this behavior by formulating review as gated distributional reweighting, proving that AI self-gating degenerates to ungated self-training under a self-confirming acceptance condition, and giving a spectral analysis of representation-level covariance concentration under recursive retraining. These results suggest that stable recursive code LLM training requires exogenous verification rather than model-coupled self-review.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Code and instruction data synthesis, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 45 of its 63 references that are in the map. First posted on arXiv 26 Jun 2026.
Builds on
45 of 63 references.
References · 45
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2021Evaluating Large Language Models Trained on Codecited by 8,320
- 2021Program Synthesis with Large Language Modelscited by 3,637
- 2023Code Llama: Open Foundation Models for Codecited by 2,139
- 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
- 2022Competition-Level Code Generation with AlphaCodecited by 1,462
- 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,391
- 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,310
- 2024Qwen2.5-Coder Technical Reportcited by 1,203
- 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,181
- 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 885
- 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 733
33 more
- 2019Fast Transformer Decoding: One Write-Head is All You Needcited by 717
- 2024LLM Evaluators Recognize and Favor Their Own Generationscited by 686
- 2024StarCoder 2 and The Stack v2: The Next Generationcited by 502
- 2022InCoder: A Generative Model for Code Infilling and Synthesiscited by 498
- 2020CodeBLEU: a Method for Automatic Evaluation of Code Synthesiscited by 479
- 2022The Stack: 3 TB of permissively licensed source codecited by 359
- 2022CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learningcited by 358
- 2023The Curse of Recursion: Training on Generated Data Makes Models Forgetcited by 356
- 2023CodeT5+: Open Code Large Language Models for Code Understanding and Generationcited by 333
- 2024DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligencecited by 331
- 2023The Impact of AI on Developer Productivity: Evidence from GitHub Copilotcited by 274
- 2023Magicoder: Empowering Code Generation with OSS-Instructcited by 256
- 2022Efficient Training of Language Models to Fill in the Middlecited by 255
- 2024OpenCodeInterpreter: Integrating Code Generation with Execution and Refinementcited by 247
- 2023SantaCoder: don't reach for the stars!cited by 170
- 2024RLEF: Grounding Code LLMs in Execution Feedback with Reinforcement Learningcited by 151
- 2022MultiPL-E: A Scalable and Extensible Approach to Benchmarking Neural Code Generationcited by 136
- 2024A Tale of Tails: Model Collapse as a Change of Scaling Lawscited by 130
- 2024OpenCoder: The Open Cookbook for Top-Tier Code Large Language Modelscited by 130
- 2024LLM-Rubric: A Multidimensional, Calibrated Approach to Automated Evaluation of Natural Language Textscited by 87
- 2024How Bad is Training on Synthetic Data? A Statistical Analysis of Language Model Collapsecited by 79
- 2024SelfCodeAlign: Self-Alignment for Code Generationcited by 62
- 2024CodeJudge: Evaluating Code Generation with Large Language Modelscited by 44
- 2024Self-Consuming Generative Models with Curated Data Provably Optimize Human Preferencescited by 39
- 2024Self-Correcting Self-Consuming Loops for Generative Model Trainingcited by 37
- 2024CodeJudge-Eval: Can Large Language Models be Good Judges in Code Understanding?cited by 30
- 2024How to Synthesize Text Data without Model Collapse?cited by 25
- 2024Rate of Model Collapse in Recursive Trainingcited by 25
- 2024Benchmarks and Metrics for Evaluations of Code Generation: A Critical Reviewcited by 20
- 2025A Theoretical Perspective: How to Prevent Model Collapse in Self-consuming Training Loopscited by 16
- 2025Rethinking Code Review Workflows with LLM Assistance: An Empirical Studycited by 10
- 2025Does AI Code Review Lead to Code Changes? A Case Study of GitHub Actionscited by 7
- 2026SWE-PRBench: Benchmarking AI Code Review Quality Against Pull Request Feedbackcited by 2
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026Self-Improving AI Coding Agents Through Accumulated Behavioral Rules: A Closed-Loop Framework
- 2026More Code, Less Reuse: Investigating Code Quality and Reviewer Sentiment towards AI-generated Pull Requests
- 2026When AI Reviews Train AI Reviewers: Scientific-Judgment Collapse and Mitigation
- 2024LLM Critics Help Catch LLM Bugs
- 2026From Code Review to Code Critique: Intent, Drift, and Spotlight for AI-Generated Diffs at Scale
- 2026From Human-Centric to Agentic Code Review: The Impact of Different Generations of Generative AI Technology on Review Quality
- 2026Early-Stage Prediction of Review Effort in AI-Generated Pull Requests
- 2025Artificial or Just Artful? Do LLMs Bend the Rules in Programming? ↗