mlsanityData to 7 Oct 2026

The Debugging Decay Index: Rethinking Debugging Strategies for Code LLMs

arXiv 2506.18403 ↗First posted 23 Jun 2025cs.SE cs.AIcited by 5

Not on the map. Why?

Abstract (arXiv)

The effectiveness of AI debugging follows a predictable exponential decay pattern; most models lose 60-80% of their debugging capability within just 2-3 attempts, despite iterative debugging being a critical capability for practical code generation systems. We introduce the Debugging Decay Index (DDI), a mathematical framework that quantifies when debugging becomes ineffective and predicts intervention points. Our strategic fresh start approach shifts from exploitation to exploration at strategic points in the debugging process, demonstrating that well-timed interventions can rescue the effectiveness of debugging. DDI reveals a fundamental limitation in current AI debugging and provides the first quantitative framework for optimising iterative code generation strategies.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 23 of its 36 references that are in the map. First posted on arXiv 23 Jun 2025.

Builds on

23 of 36 references.

References · 23

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  3. 2021Program Synthesis with Large Language Modelscited by 3,509
  4. 2022Competition-Level Code Generation with AlphaCodecited by 1,427
  5. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,332
  6. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,154
  7. 2023Teaching Large Language Models to Self-Debugcited by 963
  8. 2024A Survey on Large Language Models for Code Generationcited by 646
  9. 2020CodeBLEU: a Method for Automatic Evaluation of Code Synthesiscited by 468
  10. 2022CodeT: Code Generation with Generated Testscited by 369
  11. 2025The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexitycited by 298
  12. 2024MapCoder: Multi-Agent Code Generation for Competitive Problem Solvingcited by 177
11 more
  1. 2024Debug like a Human: A Large Language Model Debugger via Verifying Runtime Execution Step-by-stepcited by 123
  2. 2022Natural Language to Code Translation with Executioncited by 122
  3. 2024Evaluating Language Models for Efficient Code Generationcited by 69
  4. 2025Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineeringcited by 64
  5. 2024A Survey on Evaluating Large Language Models in Code Generation Taskscited by 43
  6. 2024CodeJudge: Evaluating Code Generation with Large Language Modelscited by 41
  7. 2025Revisit Self-Debugging with Self-Generated Tests for Code Generationcited by 21
  8. 2024Benchmarks and Metrics for Evaluations of Code Generation: A Critical Reviewcited by 19
  9. 2022Aligning Offline Metrics and Human Judgments of Value for Code Generation Modelscited by 14
  10. 2024CoderUJB: An Executable and Unified Java Benchmark for Practical Programming Scenarioscited by 11
  11. 2024Top Pass: Improve Code Generation by Pass@k-Maximized Code Rankingcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 5.

5 papers.

  1. 2026Code as Agent Harnesscited by 40
  2. 2026Procedural Refinement by LLM-driven Algorithmic Debugging for ARC-AGI-2cited by 2
  3. 2026Adaptive Hierarchical Evaluation of LLMs and SAST tools for CWE Prediction in Pythoncited by 1
  4. 2026From Evaluation to Optimisation: Hierarchy-Aware Training Signals for CWE Prediction in Pythoncited by 0
  5. 2026Compiler-Guided Adaptive Proof Search with Cross-Model Synergy on Context-Dependent Theorem Provingcited by 0