# The Debugging Decay Index: Rethinking Debugging Strategies for Code LLMs

arXiv: https://arxiv.org/abs/2506.18403 · HTML: https://mlsanity.com/paper/2506.18403/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 36 references that are in the map.
- First posted on arXiv 23 Jun 2025.

## Builds on (most cited first, 23 of 23)

- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Program Synthesis with Large Language Models](https://mlsanity.com/paper/2108.07732/) (2021)
- [Competition-Level Code Generation with AlphaCode](https://mlsanity.com/paper/2203.07814/) (2022)
- [Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation](https://mlsanity.com/paper/2305.01210/) (2023)
- [DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence](https://mlsanity.com/paper/2401.14196/) (2024)
- [Teaching Large Language Models to Self-Debug](https://mlsanity.com/paper/2304.05128/) (2023)
- [A Survey on Large Language Models for Code Generation](https://mlsanity.com/paper/2406.00515/) (2024)
- [CodeBLEU: a Method for Automatic Evaluation of Code Synthesis](https://mlsanity.com/paper/2009.10297/) (2020)
- [CodeT: Code Generation with Generated Tests](https://mlsanity.com/paper/2207.10397/) (2022)
- [The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity](https://mlsanity.com/paper/2506.06941/) (2025)
- [MapCoder: Multi-Agent Code Generation for Competitive Problem Solving](https://mlsanity.com/paper/2405.11403/) (2024)
- [Debug like a Human: A Large Language Model Debugger via Verifying Runtime Execution Step-by-step](https://mlsanity.com/paper/2402.16906/) (2024)
- [Natural Language to Code Translation with Execution](https://mlsanity.com/paper/2204.11454/) (2022)
- [Evaluating Language Models for Efficient Code Generation](https://mlsanity.com/paper/2408.06450/) (2024)
- [Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering](https://mlsanity.com/paper/2502.06193/) (2025)
- [A Survey on Evaluating Large Language Models in Code Generation Tasks](https://mlsanity.com/paper/2408.16498/) (2024)
- [CodeJudge: Evaluating Code Generation with Large Language Models](https://mlsanity.com/paper/2410.02184/) (2024)
- [Revisit Self-Debugging with Self-Generated Tests for Code Generation](https://mlsanity.com/paper/2501.12793/) (2025)
- [Benchmarks and Metrics for Evaluations of Code Generation: A Critical Review](https://mlsanity.com/paper/2406.12655/) (2024)
- [Aligning Offline Metrics and Human Judgments of Value for Code Generation Models](https://mlsanity.com/paper/2210.16494/) (2022)
- [CoderUJB: An Executable and Unified Java Benchmark for Practical Programming Scenarios](https://mlsanity.com/paper/2403.19287/) (2024)
- [Top Pass: Improve Code Generation by Pass@k-Maximized Code Ranking](https://mlsanity.com/paper/2408.05715/) (2024)

## Built on by (most cited first, 5 of 5)

- [Code as Agent Harness](https://mlsanity.com/paper/2605.18747/) (2026)
- [Procedural Refinement by LLM-driven Algorithmic Debugging for ARC-AGI-2](https://mlsanity.com/paper/2603.20334/) (2026)
- [Adaptive Hierarchical Evaluation of LLMs and SAST tools for CWE Prediction in Python](https://mlsanity.com/paper/2601.01320/) (2026)
- [From Evaluation to Optimisation: Hierarchy-Aware Training Signals for CWE Prediction in Python](https://mlsanity.com/paper/2607.21069/) (2026)
- [Compiler-Guided Adaptive Proof Search with Cross-Model Synergy on Context-Dependent Theorem Proving](https://mlsanity.com/paper/2608.18084/) (2026)

## Abstract (arXiv)

The effectiveness of AI debugging follows a predictable exponential decay pattern; most models lose 60-80% of their debugging capability within just 2-3 attempts, despite iterative debugging being a critical capability for practical code generation systems. We introduce the Debugging Decay Index (DDI), a mathematical framework that quantifies when debugging becomes ineffective and predicts intervention points. Our strategic fresh start approach shifts from exploitation to exploration at strategic points in the debugging process, demonstrating that well-timed interventions can rescue the effectiveness of debugging. DDI reveals a fundamental limitation in current AI debugging and provides the first quantitative framework for optimising iterative code generation strategies.