mlsanityData to 7 Oct 2026

What Makes a Good Bug Report for an AI Agent?

arXiv 2607.07593 ↗First posted 8 Jul 2026cs.SEcited by 1

On the map: LLM agents and tool use → Software engineering coding agents

Abstract (arXiv)

Automated program repair (APR) agents are transitioning from research benchmarks to developer workflows, yet they still begin with bug reports written for human developers. While decades of research have established what makes a good bug report for humans (e.g., steps to reproduce, stack traces), it remains unclear whether these features transfer to LLM-based agents. We study this question in two analyses. First, we use statistical modeling to examine associations between 27 bug-report features and repair success across 433 SWE-bench Verified issues attempted by 87 repair agents. We find that fix suggestions, reproduction scripts, repository source code, and localization info are associated with higher resolution likelihood, while longer reports are associated with lower odds. Second, we conduct controlled ablations across 2 models and 17 problem-statement mutations on SWE-bench Pro, varying the information available to an agent while holding the underlying task fixed. We remove or isolate selected bug-report content, delete fault-localization cues, and test structural changes that flatten lists or remove section headers. We find that both models depend on localization cues and expected behavior, and that structural changes alone can reduce solve rates, even without removing any content. The two models diverge in how they handle missing information: Qwen searches more widely and can exhaust its turn budget, while Gemma commits to a plausible interpretation early and patches on it. Our findings indicate that a good bug report for an agent overlaps with, but is not identical to, a good report for a human: agents benefit most from concrete, executable, and well-localized information, whereas some qualities long emphasized for human readers, such as natural language steps to reproduce and readable descriptions, contribute little or even correlate with lower success.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 14 of its 38 references that are in the map. First posted on arXiv 8 Jul 2026.

Builds on

14 of 38 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025Agentic Program Repair from Test Failures at Scale: A Neuro-symbolic approach with static analysis and test execution feedback24 Jul
  2. 2025Evaluating Agent-based Program Repair at Google13 Jan
  3. 2024Large Language Model-Based Agents for Software Engineering: A Survey4 Sep
  4. 2024HumanEvalComm: Benchmarking the Communication Competence of Code Generation for LLMs and LLM Agent31 May
  5. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering6 May

Other references · 9

  1. 2025Qwen3 Technical Reportcited by 8,195
  2. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  3. 2024AutoCodeRover: Autonomous Program Improvementcited by 272
  4. 2025SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?cited by 254
  5. 2024SWE-Bench+: Enhanced Coding Benchmark for LLMscited by 95
  6. 2025The SWE-Bench Illusion: When State-of-the-Art LLMs Remember Instead of Reasoncited by 47
  7. 2019Assessing the Quality of the Steps to Reproduce in Bug Reportscited by 13
  8. 2025Past, Present, and Future of Bug Tracking in the Generative AI Eracited by 4
  9. 2021Why are Some Bugs Non-Reproducible? An Empirical Investigation using Data Fusioncited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026Writing Bug Reports for Software Repair Agents: What Information Matters Most?
  2. 2025Understanding Automated Program Repair Agents Through the Lens of Traceability: An Empirical Study
  3. 2024An Empirical Study on LLM-based Agents for Automated Bug Fixing
  4. 2026Dynamic Cogeneration of Bug Reproduction Test in Agentic Program Repair
  5. 2026EviACT: An Evidence-to-Action Framework for Agentic Program Repair
  6. 2026SelfHeal: Empirical Fix Pattern Analysis and Bug Repair in LLM Agents
  7. 2026Active-SWE: Benchmarking Coding Agents for Proactive Bug Fixing without Issue Reports
  8. 2026Beyond Bug Fixes: An Empirical Investigation of Post-Merge Code Quality Issues in Agent-Generated Pull Requests