# Assessing the Quality of the Steps to Reproduce in Bug Reports

arXiv: https://arxiv.org/abs/1906.07107 · HTML: https://mlsanity.com/paper/1906.07107/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Jun 2019.

## Built on by (most cited first, 13 of 13)

- [BountyBench: Dollar Impact of AI Agent Attackers and Defenders on Real-World Cybersecurity Systems](https://mlsanity.com/paper/2505.15216/) (2025)
- [Agentic Bug Reproduction for Effective Automated Program Repair at Google](https://mlsanity.com/paper/2502.01821/) (2025)
- [On the Impact of AGENTS.md Files on the Efficiency of AI Coding Agents](https://mlsanity.com/paper/2601.20404/) (2026)
- [Evaluating and Improving Automated Repository-Level Rust Issue Resolution with LLM-based Agents](https://mlsanity.com/paper/2602.22764/) (2026)
- [Past, Present, and Future of Bug Tracking in the Generative AI Era](https://mlsanity.com/paper/2510.08005/) (2025)
- [Can We Enhance Bug Report Quality Using LLMs?: An Empirical Study of LLM-Based Bug Report Generation](https://mlsanity.com/paper/2504.18804/) (2025)
- [Combining Language and App UI Analysis for the Automated Assessment of Bug Reproduction Steps](https://mlsanity.com/paper/2502.04251/) (2025)
- [ImproBR: Bug Report Improver Using LLMs](https://mlsanity.com/paper/2604.26142/) (2026)
- [What Makes a Good Bug Report for an AI Agent?](https://mlsanity.com/paper/2607.07593/) (2026)
- [Writing Bug Reports for Software Repair Agents: What Information Matters Most?](https://mlsanity.com/paper/2607.09553/) (2026)
- [AURORA: Navigating UI Tarpits via Automated Neural Screen Understanding](https://mlsanity.com/paper/2404.01240/) (2024)
- [FeedAIde: Guiding App Users to Submit Rich Feedback Reports by Asking Context-Aware Follow-Up Questions](https://mlsanity.com/paper/2603.04244/) (2026)
- [How Do LLMs Read Bug Reports? An Empirical Study of Attention in LLMs for Automated Program Repair](https://mlsanity.com/paper/2607.25873/) (2026)

## Abstract (arXiv)

A major problem with user-written bug reports, indicated by developers and documented by researchers, is the (lack of high) quality of the reported steps to reproduce the bugs. Low-quality steps to reproduce lead to excessive manual effort spent on bug triage and resolution. This paper proposes Euler, an approach that automatically identifies and assesses the quality of the steps to reproduce in a bug report, providing feedback to the reporters, which they can use to improve the bug report. The feedback provided by Euler was assessed by external evaluators and the results indicate that Euler correctly identified 98% of the existing steps to reproduce and 58% of the missing ones, while 73% of its quality annotations are correct.