# Agentic Bug Reproduction for Effective Automated Program Repair at Google

arXiv: https://arxiv.org/abs/2502.01821 · HTML: https://mlsanity.com/paper/2502.01821/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use.
- Builds on 20 of its 46 references that are in the map.
- First posted on arXiv 3 Feb 2025.

## Placed in

- LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/)

## Builds on (most cited first, 20 of 20)

- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [SWE-bench Multimodal: Do AI Systems Generalize to Visual Software Domains?](https://mlsanity.com/paper/2410.03859/) (2024)
- [An Empirical Evaluation of Using Large Language Models for Automated Unit Test Generation](https://mlsanity.com/paper/2302.06527/) (2023)
- [SWT-Bench: Testing and Validating Real-World Bug-Fixes with Code Agents](https://mlsanity.com/paper/2406.12952/) (2024)
- [Large Language Models are Few-shot Testers: Exploring LLM-based General Bug Reproduction](https://mlsanity.com/paper/2209.11515/) (2022)
- [ChatUniTest: A Framework for LLM-Based Test Generation](https://mlsanity.com/paper/2305.04764/) (2023)
- [CodeR: Issue Resolving with Multi-Agent and Task Graphs](https://mlsanity.com/paper/2406.01304/) (2024)
- [Impact of Code Language Models on Automated Program Repair](https://mlsanity.com/paper/2302.05020/) (2023)
- [No More Manual Tests? Evaluating and Improving ChatGPT for Unit Test Generation](https://mlsanity.com/paper/2305.04207/) (2023)
- [Large Language Models for Test-Free Fault Localization](https://mlsanity.com/paper/2310.01726/) (2023)
- [AgentFL: Scaling LLM-based Fault Localization to Project-Level Context](https://mlsanity.com/paper/2403.16362/) (2024)
- [Evaluating Agent-based Program Repair at Google](https://mlsanity.com/paper/2501.07531/) (2025)
- [TDD-Bench Verified: Can LLMs Generate Tests for Issues Before They Get Resolved?](https://mlsanity.com/paper/2412.02883/) (2024)
- [Nuances are the Key: Unlocking ChatGPT to Find Failure-Inducing Tests with Differential Prompting](https://mlsanity.com/paper/2304.11686/) (2023)
- [Elixir: Effective object-oriented program repair](https://mlsanity.com/paper/2112.10915/) (2021)
- [Precise Condition Synthesis for Program Repair](https://mlsanity.com/paper/1608.07754/) (2016)
- [Assessing the Quality of the Steps to Reproduce in Bug Reports](https://mlsanity.com/paper/1906.07107/) (2019)

## Built on by (most cited first, 22 of 22)

- [Understanding Software Engineering Agents: A Study of Thought-Action-Result Trajectories](https://mlsanity.com/paper/2506.18824/) (2025)
- [From Code Foundation Models to Agents and Applications: A Comprehensive Survey and Practical Guide to Code Intelligence](https://mlsanity.com/paper/2511.18538/) (2025)
- [Seeing is Fixing: Cross-Modal Reasoning with Multimodal LLMs for Visual Software Issue Fixing](https://mlsanity.com/paper/2506.16136/) (2025)
- [The Rise of AI Teammates in Software Engineering (SE) 3.0: How Autonomous Coding Agents Are Reshaping Software Engineering](https://mlsanity.com/paper/2507.15003/) (2025)
- [AgentStepper: Interactive Debugging of Software Development Agents](https://mlsanity.com/paper/2602.06593/) (2026)
- [Agentic Program Repair from Test Failures at Scale: A Neuro-symbolic approach with static analysis and test execution feedback](https://mlsanity.com/paper/2507.18755/) (2025)
- [Agentic Software Issue Resolution with Large Language Models: A Survey](https://mlsanity.com/paper/2512.22256/) (2025)
- [Dynamic Cogeneration of Bug Reproduction Test in Agentic Program Repair](https://mlsanity.com/paper/2601.19066/) (2026)
- [SWE-Doctor: Guiding Software Engineering Agents with Runtime Diagnosis from Multi-Faceted Bug Reproduction Tests](https://mlsanity.com/paper/2607.00990/) (2026)
- [Past, Present, and Future of Bug Tracking in the Generative AI Era](https://mlsanity.com/paper/2510.08005/) (2025)
- [Execution-State-Aware LLM Reasoning for Automated Proof-of-Vulnerability Generation](https://mlsanity.com/paper/2602.13574/) (2026)
- [ExecCritic: Learn to Test, Test to Improve for Coding Agents](https://mlsanity.com/paper/2609.09133/) (2026)
- [LinkAnchor: An Autonomous LLM-Based Agent for Issue-to-Commit Link Recovery](https://mlsanity.com/paper/2508.12232/) (2025)
- [Beyond Accuracy: Behavioral Dynamics of Agentic Multi-Hunk Repair](https://mlsanity.com/paper/2511.11012/) (2025)
- [EvoOtter: Evolutionary Reproduction Test Generator](https://mlsanity.com/paper/2607.02854/) (2026)
- [Agentic Reinforcement Learning for Real-World Code Repair](https://mlsanity.com/paper/2510.22075/) (2025)
- [Evaluating LLM Agents on Automated Software Analysis Tasks](https://mlsanity.com/paper/2604.11270/) (2026)
- [DPIAgent: Divide, Protocol, Isolate for Agentic Reproduction Test Generation](https://mlsanity.com/paper/2608.23341/) (2026)
- [Better Understanding, Better Fixes? A Study of Hallucination in LLM-based Automated Program Repair](https://mlsanity.com/paper/2609.04909/) (2026)
- [WitnessGym: Benchmarking Coding Agents on the Construction of Bug Witnesses](https://mlsanity.com/paper/2609.36635/) (2026)
- [Can LLMs Fix It Without Code? Toward Automated Verification of No-Code Bug Fixes](https://mlsanity.com/paper/2610.11963/) (2026)
- [SWEPatchTest: Evaluating Software Tests Beyond Reproduction](https://mlsanity.com/paper/2610.18004/) (2026)

## Abstract (arXiv)

Bug reports often lack sufficient detail for developers to reproduce and fix the underlying defects. Bug Reproduction Tests (BRTs), tests that fail when the bug is present and pass when it has been resolved, are crucial for debugging, but they are rarely included in bug reports, both in open-source and in industrial settings. Thus, automatically generating BRTs from bug reports has the potential to accelerate the debugging process and lower time to repair. This paper investigates automated BRT generation within an industry setting, specifically at Google, focusing on the challenges of a large-scale, proprietary codebase and considering real-world industry bugs extracted from Google's internal issue tracker. We adapt and evaluate a state-of-the-art BRT generation technique, LIBRO, and present our agent-based approach, BRT Agent, which makes use of a fine-tuned Large Language Model (LLM) for code editing. Our BRT Agent significantly outperforms LIBRO, achieving a 28% plausible BRT generation rate, compared to 10% by LIBRO, on 80 human-reported bugs from Google's internal issue tracker. We further investigate the practical value of generated BRTs by integrating them with an Automated Program Repair (APR) system at Google. Our results show that providing BRTs to the APR system results in 30% more bugs with plausible fixes. Additionally, we introduce Ensemble Pass Rate (EPR), a metric which leverages the generated BRTs to select the most promising fixes from all fixes generated by APR system. Our evaluation on EPR for Top-K and threshold-based fix selections demonstrates promising results and trade-offs. For example, EPR correctly selects a plausible fix from a pool of 20 candidates in 70% of cases, based on its top-1 ranking.