# Explainable Automated Debugging via Large Language Model-driven Scientific Debugging

arXiv: https://arxiv.org/abs/2304.02195 · HTML: https://mlsanity.com/paper/2304.02195/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Apr 2023.

## Built on by (most cited first, 30 of 40)

- [Large Language Models for Software Engineering: A Systematic Literature Review](https://mlsanity.com/paper/2308.10620/) (2023)
- [RepairAgent: An Autonomous, LLM-Based Agent for Program Repair](https://mlsanity.com/paper/2403.17134/) (2024)
- [Large Language Model-Based Agents for Software Engineering: A Survey](https://mlsanity.com/paper/2409.02977/) (2024)
- [ClassEval: A Manually-Crafted Benchmark for Evaluating LLMs on Class-level Code Generation](https://mlsanity.com/paper/2308.01861/) (2023)
- [LLM-Based Multi-Agent Systems for Software Engineering: Literature Review, Vision and the Road Ahead](https://mlsanity.com/paper/2404.04834/) (2024)
- [From LLMs to LLM-based Agents for Software Engineering: A Survey of Current, Challenges and Future](https://mlsanity.com/paper/2408.02479/) (2024)
- [NExT: Teaching Large Language Models to Reason about Code Execution](https://mlsanity.com/paper/2404.14662/) (2024)
- [Reducing Cost of LLM Agents with Trajectory Reduction](https://mlsanity.com/paper/2509.23586/) (2025)
- [AI Agentic Programming: A Survey of Techniques, Challenges, and Opportunities](https://mlsanity.com/paper/2508.11126/) (2025)
- [XAI meets LLMs: A Survey of the Relation between Explainable AI and Large Language Models](https://mlsanity.com/paper/2407.15248/) (2024)
- [An Empirical Study on LLM-based Agents for Automated Bug Fixing](https://mlsanity.com/paper/2411.10213/) (2024)
- [Beyond pip install: Evaluating LLM Agents for the Automated Installation of Python Projects](https://mlsanity.com/paper/2412.06294/) (2024)
- [EvoC2Rust: A Skeleton-guided Framework for Project-Level C-to-Rust Translation](https://mlsanity.com/paper/2508.04295/) (2025)
- [Large Language Models (LLMs) for Source Code Analysis: applications, models and datasets](https://mlsanity.com/paper/2503.17502/) (2025)
- [GPT-Based Models Meet Simulation: How to Efficiently Use Large-Scale Pre-Trained Language Models Across Simulation Tasks](https://mlsanity.com/paper/2306.13679/) (2023)
- [Towards Autonomous Testing Agents via Conversational Large Language Models](https://mlsanity.com/paper/2306.05152/) (2023)
- [ChatDBG: Augmenting Debugging with Large Language Models](https://mlsanity.com/paper/2403.16354/) (2024)
- [Hallucination to Consensus: Multi-Agent LLMs for End-to-End JUnit Test Generation](https://mlsanity.com/paper/2506.02943/) (2025)
- [Challenges in Testing Large Language Model Based Software: A Faceted Taxonomy](https://mlsanity.com/paper/2503.00481/) (2025)
- [LLM-Based Agentic Systems for Software Engineering: Challenges and Opportunities](https://mlsanity.com/paper/2601.09822/) (2026)
- [Agent That Debugs: Dynamic State-Guided Vulnerability Repair](https://mlsanity.com/paper/2504.07634/) (2025)
- [Tuning LLM-based Code Optimization via Meta-Prompting: An Industrial Perspective](https://mlsanity.com/paper/2508.01443/) (2025)
- [Code Copycat Conundrum: Demystifying Repetition in LLM-based Code Generation](https://mlsanity.com/paper/2504.12608/) (2025)
- [Past, Present, and Future of Bug Tracking in the Generative AI Era](https://mlsanity.com/paper/2510.08005/) (2025)
- [Will Code Remain a Relevant User Interface for End-User Programming with Generative AI Models?](https://mlsanity.com/paper/2311.00382/) (2023)
- [DynaFix: Iterative Automated Program Repair Driven by Execution-Level Dynamic Information](https://mlsanity.com/paper/2512.24635/) (2025)
- [S3LLM: Large-Scale Scientific Software Understanding with LLMs using Source, Metadata, and Document](https://mlsanity.com/paper/2403.10588/) (2024)
- [Generative transformations and patterns in LLM-native approaches for software verification and falsification](https://mlsanity.com/paper/2404.09384/) (2024)
- [Redundancy and Concept Analysis for Code-trained Language Models](https://mlsanity.com/paper/2305.00875/) (2023)
- [Assessing the Code Clone Detection Capability of Large Language Models](https://mlsanity.com/paper/2407.02402/) (2024)

## Abstract (arXiv)

Automated debugging techniques have the potential to reduce developer effort in debugging, and have matured enough to be adopted by industry. However, one critical issue with existing techniques is that, while developers want rationales for the provided automatic debugging results, existing techniques are ill-suited to provide them, as their deduction process differs significantly from that of human developers. Inspired by the way developers interact with code when debugging, we propose Automated Scientific Debugging (AutoSD), a technique that given buggy code and a bug-revealing test, prompts large language models to automatically generate hypotheses, uses debuggers to actively interact with buggy code, and thus automatically reach conclusions prior to patch generation. By aligning the reasoning of automated debugging more closely with that of human developers, we aim to produce intelligible explanations of how a specific patch has been generated, with the hope that the explanation will lead to more efficient and accurate developer decisions. Our empirical analysis on three program repair benchmarks shows that AutoSD performs competitively with other program repair baselines, and that it can indicate when it is confident in its results. Furthermore, we perform a human study with 20 participants, including six professional developers, to evaluate the utility of explanations from AutoSD. Participants with access to explanations could judge patch correctness in roughly the same time as those without, but their accuracy improved for five out of six real-world bugs studied: 70% of participants answered that they wanted explanations when using repair tools, while 55% answered that they were satisfied with the Scientific Debugging presentation.