Beyond Accuracy: Behavioral Dynamics of Agentic Multi-Hunk Repair
On the map: LLM agents and tool use → Software engineering coding agents
Abstract (arXiv)
Automated program repair has traditionally focused on single-hunk defects, overlooking multi-hunk bugs that are prevalent in real-world systems. Repairing these bugs requires coordinated edits across multiple, disjoint code regions, posing substantially greater challenges. We present the first systematic study of LLM-driven coding agents (Claude Code, Codex, Gemini-cli, and Qwen Code) on this task. We evaluate these four state-of-the-art agents on 404 multi-hunk bugs from the PolyHunk dataset, yielding 1,616 repair trajectories for large-scale behavioral analysis. We employ fine-grained metrics to assess localization, repair accuracy, regression behavior, and operational dynamics across agents. We find that localization capability varies substantially, with Codex achieving the highest success rate (75.3%) and Qwen Code the lowest (40.4%). Repair accuracy also differs widely, ranging from 26.98% (Qwen Code) to 92.82% (Claude Code), and consistently declines with increasing bug dispersion and complexity (hunk divergence and spatial proximity). High-performing agents (Claude Code and Codex) demonstrate superior semantic consistency, achieving positive average regression reduction, whereas lower-performing agents often introduce new test failures. Notably, agents do not fail fast; failed repairs consume substantially more resources (33%-440% more input tokens) and require longer execution time (35%-330%). Additionally, we developed Maple to provide agents with repository-level context. Empirical results show that Maple improves repair accuracy of Gemini-cli by ~21% through enhanced localization. By analyzing fine-grained metrics and trajectory-level analysis, this study moves beyond accuracy to explain how coding agents localize, reason, and act during multi-hunk repair. Our findings underscore the impact of bug divergence and spatial proximity on multi-hunk repair success for coding agents.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 25 of its 54 references that are in the map. First posted on arXiv 14 Nov 2025.
Builds on
25 of 54 references.
Closest ancestors · 12
Same direction, newest first
- 2025Understanding Software Engineering Agents: A Study of Thought-Action-Result Trajectories23 Jun
- 2025Agentic Bug Reproduction for Effective Automated Program Repair at Google3 Feb
- 2025Evaluating Agent-based Program Repair at Google13 Jan
- 2024You Name It, I Run It: An LLM Agent to Execute Tests of Arbitrary Projects13 Dec
- 2024Beyond pip install: Evaluating LLM Agents for the Automated Installation of Python Projects9 Dec
- 2024Debug Smarter, Not Harder: AI Agents for Error Resolution in Computational Notebooks18 Oct
- 2024MarsCode Agent: AI-native Automated Bug Fixing2 Sep
- 2024SpecRover: Code Intent Extraction via LLMs5 Aug
- 2024OpenHands: An Open Platform for AI Software Developers as Generalist Agents23 Jul
- 2024SWT-Bench: Testing and Validating Real-World Bug-Fixes with Code Agents18 Jun
- 2024MAGIS: LLM-Based Multi-Agent Framework for GitHub Issue Resolution26 Mar
- 2024RepairAgent: An Autonomous, LLM-Based Agent for Program Repair25 Mar
Other references · 13
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
- 2021A Syntax-Guided Edit Decoder for Neural Program Repaircited by 57
- 2021Neural Program Repair with Execution-based Backpropagationcited by 50
- 2018Nopol: Automatic Repair of Conditional Statement Bugs in Java Programscited by 32
- 2025Issue2Test: Generating Reproducing Test Cases from Issue Reportscited by 18
- 2023ITER: Iterative Neural Repair for Multi-Location Patchescited by 16
- 2019Harnessing Evolution for Multi-Hunk Program Repaircited by 12
- 2024RepairBench: Leaderboard of Frontier Models for Program Repaircited by 9
- 2025Characterizing Multi-Hunk Patches: Divergence, Proximity, and LLM Repair Challengescited by 7
- 2024Contextual API Completion for Unseen Repositories Using LLMscited by 6
- 2021A Controlled Experiment of Different Code Representations for Learning-Based Bug Repaircited by 5
1 more
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026To Run or Not to Run: Analyzing the Cost-Effectiveness of Code Execution in LLM-Based Program Repair
- 2024An Empirical Study on LLM-based Agents for Automated Bug Fixing
- 2025Understanding Automated Program Repair Agents Through the Lens of Traceability: An Empirical Study
- 2026Coherence Collapse: Diagnosing Why Code Agents Fail After Reaching the Right Code
- 2025Understanding Code Agent Behaviour: An Empirical Study of Success and Failure Trajectories
- 2026Inside the Scaffold: A Source-Code Taxonomy of Coding Agent Architectures
- 2026Beyond Resolution Rates: Behavioral Drivers of Coding Agent Success and Failure
- 2026When Passing Tests Hides Vulnerabilities: An Empirical Study of Silent Failures in Agentic Systems