mlsanityData to 14 Oct 2026

When AI Reviews Its Own Code: Recursive Self-Training Collapse in Code LLMs

arXiv 2606.28438 ↗First posted 26 Jun 2026cs.SE cs.AIcited by 2

On the map: Synthetic data and self-improvement for LLMs → Code and instruction data synthesis

Abstract (arXiv)

Recursive self-training can degrade neural generative models when generated data is reused without fresh human data or external quality control. We study this risk in code LLMs, where AI-generated code can enter real repositories, later become training data, and create a repository-scale self-training loop. While software development traditionally interrupts this loop through pull-request review, tests, compilation, and human approval, AI coding tools now produce code faster than humans can review it, and code review itself is increasingly automated by AI systems. We therefore compare three recursive fine-tuning regimes: no review, human-style exogenous-gate review using model-independent filters such as compilation and static quality checks, and AI-self-gate review using the code LLM's own signals such as perplexity and binary self-scoring. Across multiple code LLMs and benchmarks, every tested recursive regime falls below its own starting point; exogenous filters preserve executable validity and sometimes reduce decline, while AI-self-gate filters can look strong early but later lose their filtering effect. In the clearest case, the binary self-gate enters a rubber-stamp regime where acceptance scores rise while benchmark correctness falls. We explain this behavior by formulating review as gated distributional reweighting, proving that AI self-gating degenerates to ungated self-training under a self-confirming acceptance condition, and giving a spectral analysis of representation-level covariance concentration under recursive retraining. These results suggest that stable recursive code LLM training requires exogenous verification rather than model-coupled self-review.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Code and instruction data synthesis, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 45 of its 63 references that are in the map. First posted on arXiv 26 Jun 2026.

Builds on

45 of 63 references.

References · 45

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  3. 2021Program Synthesis with Large Language Modelscited by 3,637
  4. 2023Code Llama: Open Foundation Models for Codecited by 2,139
  5. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
  6. 2022Competition-Level Code Generation with AlphaCodecited by 1,462
  7. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,391
  8. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,310
  9. 2024Qwen2.5-Coder Technical Reportcited by 1,203
  10. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,181
  11. 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 885
  12. 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 733
33 more
  1. 2019Fast Transformer Decoding: One Write-Head is All You Needcited by 717
  2. 2024LLM Evaluators Recognize and Favor Their Own Generationscited by 686
  3. 2024StarCoder 2 and The Stack v2: The Next Generationcited by 502
  4. 2022InCoder: A Generative Model for Code Infilling and Synthesiscited by 498
  5. 2020CodeBLEU: a Method for Automatic Evaluation of Code Synthesiscited by 479
  6. 2022The Stack: 3 TB of permissively licensed source codecited by 359
  7. 2022CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learningcited by 358
  8. 2023The Curse of Recursion: Training on Generated Data Makes Models Forgetcited by 356
  9. 2023CodeT5+: Open Code Large Language Models for Code Understanding and Generationcited by 333
  10. 2024DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligencecited by 331
  11. 2023The Impact of AI on Developer Productivity: Evidence from GitHub Copilotcited by 274
  12. 2023Magicoder: Empowering Code Generation with OSS-Instructcited by 256
  13. 2022Efficient Training of Language Models to Fill in the Middlecited by 255
  14. 2024OpenCodeInterpreter: Integrating Code Generation with Execution and Refinementcited by 247
  15. 2023SantaCoder: don't reach for the stars!cited by 170
  16. 2024RLEF: Grounding Code LLMs in Execution Feedback with Reinforcement Learningcited by 151
  17. 2022MultiPL-E: A Scalable and Extensible Approach to Benchmarking Neural Code Generationcited by 136
  18. 2024A Tale of Tails: Model Collapse as a Change of Scaling Lawscited by 130
  19. 2024OpenCoder: The Open Cookbook for Top-Tier Code Large Language Modelscited by 130
  20. 2024LLM-Rubric: A Multidimensional, Calibrated Approach to Automated Evaluation of Natural Language Textscited by 87
  21. 2024How Bad is Training on Synthetic Data? A Statistical Analysis of Language Model Collapsecited by 79
  22. 2024SelfCodeAlign: Self-Alignment for Code Generationcited by 62
  23. 2024CodeJudge: Evaluating Code Generation with Large Language Modelscited by 44
  24. 2024Self-Consuming Generative Models with Curated Data Provably Optimize Human Preferencescited by 39
  25. 2024Self-Correcting Self-Consuming Loops for Generative Model Trainingcited by 37
  26. 2024CodeJudge-Eval: Can Large Language Models be Good Judges in Code Understanding?cited by 30
  27. 2024How to Synthesize Text Data without Model Collapse?cited by 25
  28. 2024Rate of Model Collapse in Recursive Trainingcited by 25
  29. 2024Benchmarks and Metrics for Evaluations of Code Generation: A Critical Reviewcited by 20
  30. 2025A Theoretical Perspective: How to Prevent Model Collapse in Self-consuming Training Loopscited by 16
  31. 2025Rethinking Code Review Workflows with LLM Assistance: An Empirical Studycited by 10
  32. 2025Does AI Code Review Lead to Code Changes? A Case Study of GitHub Actionscited by 7
  33. 2026SWE-PRBench: Benchmarking AI Code Review Quality Against Pull Request Feedbackcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026Self-Improving AI Coding Agents Through Accumulated Behavioral Rules: A Closed-Loop Framework
  2. 2026More Code, Less Reuse: Investigating Code Quality and Reviewer Sentiment towards AI-generated Pull Requests
  3. 2026When AI Reviews Train AI Reviewers: Scientific-Judgment Collapse and Mitigation
  4. 2024LLM Critics Help Catch LLM Bugs
  5. 2026From Code Review to Code Critique: Intent, Drift, and Spotlight for AI-Generated Diffs at Scale
  6. 2026From Human-Centric to Agentic Code Review: The Impact of Different Generations of Generative AI Technology on Review Quality
  7. 2026Early-Stage Prediction of Review Effort in AI-Generated Pull Requests
  8. 2025Artificial or Just Artful? Do LLMs Bend the Rules in Programming? ↗