mlsanityData to 7 Oct 2026

A Quantitative and Qualitative Evaluation of LLM-Based Explainable Fault Localization

arXiv 2308.05487 ↗First posted 10 Aug 2023cs.SEcited by 50

Not on the map. Why?

Abstract (arXiv)

Fault Localization (FL), in which a developer seeks to identify which part of the code is malfunctioning and needs to be fixed, is a recurring challenge in debugging. To reduce developer burden, many automated FL techniques have been proposed. However, prior work has noted that existing techniques fail to provide rationales for the suggested locations, hindering developer adoption of these techniques. With this in mind, we propose AutoFL, a Large Language Model (LLM)-based FL technique that generates an explanation of the bug along with a suggested fault location. AutoFL prompts an LLM to use function calls to navigate a repository, so that it can effectively localize faults over a large software repository and overcome the limit of the LLM context length. Extensive experiments on 798 real-world bugs in Java and Python reveal AutoFL improves method-level acc@1 by up to 233.3% over baselines. Furthermore, developers were interviewed on their impression of AutoFL-generated explanations, showing that developers generally liked the natural language explanations of AutoFL, and that they preferred reading a few, high-quality explanations instead of many.

Key facts. Cited by 50 papers in the map (data through 7 Oct 2026). First posted on arXiv 10 Aug 2023.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 13; 2025: 21; 2026: 16.

50 papers.

Sort:

By area:

  1. 2024Agentless: Demystifying LLM-based Software Engineering Agentscited by 400
  2. 2025LocAgent: Graph-Guided LLM Agents for Code Localizationcited by 82
  3. 2024HyperAgent: Generalist Software Engineering Agents to Solve Coding Tasks at Scalecited by 61
  4. 2025Issue Localization via LLM-Driven Iterative Code Graph Searchingcited by 22
  5. 2024Beyond pip install: Evaluating LLM Agents for the Automated Installation of Python Projectscited by 21
  6. 2025Migrating Code At Scale With LLMs At Googlecited by 14
  7. 2026Where Do AI Coding Agents Fail? An Empirical Study of Failed Agentic Pull Requests in GitHubcited by 10
  8. 2024ChatDBG: Augmenting Debugging with Large Language Modelscited by 9
  9. 2024A Multi-Agent Approach to Fault Localization via Graph-Based Retrieval and Reflexioncited by 8
  10. 2025Challenges in Testing Large Language Model Based Software: A Faceted Taxonomycited by 8
  11. 2024Retrieval-Augmented Test Generation: How Far Are We?cited by 7
  12. 2024Enhancing Automated Loop Invariant Generation for Complex Programs with Large Language Modelscited by 7
38 more
  1. 2025On the Effectiveness of Large Language Models in Writing Alloy Formulascited by 7
  2. 2025GALA: Can Graph-Augmented Large Language Model Agentic Workflows Elevate Root Cause Analysis?cited by 5
  3. 2026RGFL: Reasoning Guided Fault Localization for Automated Program Repair Using Large Language Modelscited by 5
  4. 2026MASPrism: Lightweight Failure Attribution for Multi-Agent Systems Using Prefill-Stage Signalscited by 5
  5. 2024Integrating Various Software Artifacts for Better LLM-based Bug Localization and Program Repaircited by 4
  6. 2025Assessing the Impact of Code Changes on the Fault Localizability of Large Language Modelscited by 4
  7. 2025From Trace to Line: LLM Agent for Real-World OSS Vulnerability Localizationcited by 4
  8. 2025Past, Present, and Future of Bug Tracking in the Generative AI Eracited by 4
  9. 2024Order Matters! An Empirical Study on Large Language Models' Input Order Bias in Software Fault Localizationcited by 3
  10. 2025COSMosFL: Ensemble of Small Language Models for Fault Localisationcited by 3
  11. 2025Localized Calibrated Uncertainty in Code Language Modelscited by 3
  12. 2026Atropos: Improving Cost-Benefit Trade-off of LLM-based Agents under Self-Consistency with Early Termination and Model Hotswapcited by 3
  13. 2024Generative transformations and patterns in LLM-native approaches for software verification and falsificationcited by 2
  14. 2024Detection Made Easy: Potentials of Large Language Models for Solidity Vulnerabilitiescited by 2
  15. 2025LinkAnchor: An Autonomous LLM-Based Agent for Issue-to-Commit Link Recoverycited by 2
  16. 2025OBsmith: LLM-Powered JavaScript Obfuscator Testingcited by 2
  17. 2026SHERLOC: Structured Diagnostic Localization for Code Repair Agentscited by 2
  18. 2026PhoenixRepair: Rethinking Repair Strategy Exploration in Software Agentscited by 2
  19. 2025Capturing Semantic Flow of ML-based Systemscited by 1
  20. 2025CAMB: A comprehensive industrial LLM benchmark on civil aviation maintenancecited by 1
  21. 2025CANDY: Benchmarking LLMs' Limitations and Assistive Potential in Chinese Misinformation Fact-Checkingcited by 1
  22. 2025Clotho: Measuring Task-Specific Pre-Generation Test Adequacy for LLM Inputscited by 1
  23. 2025SpIDER: Spatially Informed Dense Embedding Retrieval for Software Issue Localizationcited by 1
  24. 2026Explainable Token-level Noise Filtering for LLM Fine-tuning Datasetscited by 1
  25. 2026Cost-Effective Repository Exploration for Agentic Issue Localizationcited by 1
  26. 2024Demystifying Faulty Code with LLM: Step-by-Step Reasoning for Explainable Fault Localizationcited by 0
  27. 2024Exploiting Latent Linearity in LLMs Improves Explainable Molecular Representation Learningcited by 0
  28. 2025Towards User-Centred Design of AI-Assisted Decision-Making in Law Enforcementcited by 0
  29. 2025Time Travel: LLM-Assisted Semantic Behavior Localization with Git Bisectcited by 0
  30. 2025ROAD: Reflective Optimization via Automated Debugging for Zero-Shot Agent Alignmentcited by 0
  31. 2026When AI Navigates the Fog of Warcited by 0
  32. 2026SemLoc: Structured Grounding of Free-Form LLM Reasoning for Fault Localizationcited by 0
  33. 2026LLM-Based Automated Diagnosis Of Integration Test Failures At Googlecited by 0
  34. 2026An Empirical Evaluation of Locally Deployed LLMs for Bug Detection in Python Codecited by 0
  35. 2026Bug-Report-Driven Fault Localization: Industrial Benchmarking and Lesson Learned at ABB Roboticscited by 0
  36. 2026BLAgent: Agentic RAG for File-Level Bug Localizationcited by 0
  37. 2026Ekka: Automated Diagnosis of Silent Errors in LLM Inferencecited by 0
  38. 2026HyperFL: Query-Adaptive Representation Learning for Software Fault Localizationcited by 0