LLM-Based Agents for Identifying Bug-Introducing Commits
On the map: LLM agents and tool use → Software engineering coding agents
Abstract (arXiv)
'Sliwerski, Zimmermann, and Zeller (SZZ) just won the 2026 ACM SIGSOFT Impact Award for asking: When do changes induce fixes? Their paper from 2005 served as the foundation for a wide array of approaches aimed at identifying bug-introducing changes (or commits) from fix commits in software repositories. But even after two decades of progress, the best-performing approach from 2025 yields a modest increase of 10 percentage points in F1-score on the most popular Linux kernel dataset. In this paper, we uncover how and why LLM-based agents can substantially advance the state-of-the-art in identifying bug-introducing commits from fix commits. We propose a simple agentic workflow based on searching a set of candidate commits and find that it raises the F1-score from 0.64 to 0.81 on the most popular Linux kernel dataset, a bigger jump than between the original 2005 method (0.54) and the previous SOTA (0.64). We also uncover why agents are so successful: They derive short greppable patterns from the fix commit diff and message and use them to effectively search and find bug-introducing commits in large candidate sets. Finally, we also discuss how these insights might enable further progress in root cause understanding and repair.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 8 of its 29 references that are in the map. First posted on arXiv 31 Mar 2026.
Builds on
8 of 29 references.
Closest ancestors · 1
Same direction, newest first
Other references · 7
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
- 2024CRUXEval: A Benchmark for Code Reasoning, Understanding and Executioncited by 300
- 2023Uncovering the Limits of Machine Learning for Automatic Vulnerability Detectioncited by 30
- 2024Top Score on the Wrong Exam: On Benchmarking in Machine Learning for Vulnerability Detectioncited by 28
- 2021Evaluating SZZ Implementations Through a Developer-informed Oraclecited by 5
- 2025Vulnerability-Affected Versions Identification: How Far Are We?cited by 3
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026AgentSZZ: Teaching the LLM Agent to Play Detective with Bug-Inducing Commits
- 2026AgenticSZZ: Temporal Knowledge Graph-Guided Agentic Bug-Inducing Commit Identification
- 2026SelfHeal: Empirical Fix Pattern Analysis and Bug Repair in LLM Agents
- 2026When Agents Fail: A Comprehensive Study of Bugs in LLM Agents with Automated Labeling
- 2026Outrunning LLM Cutoffs: A Live Kernel Crash Resolution Benchmark for All
- 2025From LLMs to Agents: A Comparative Evaluation of LLMs and LLM-based Agents in Security Patch Detection
- 2026MAS-SZZ: Multi-Agentic SZZ Algorithm for Vulnerability-Inducing Commit Identification ↗
- 2026Detecting Vulnerability-Inducing Commits via Multi-Stage Reasoning with LLM-Based Agents