Do Autonomous Agents Contribute Test Code? A Study of Tests in Agentic Pull Requests
On the map: LLM agents and tool use → Software engineering coding agents
Abstract (arXiv)
Testing is a critical practice for ensuring software correctness and long-term maintainability. As agentic coding tools increasingly submit pull requests (PRs), it becomes essential to understand how testing appears in these agent-driven workflows. Using the AIDev dataset, we present an empirical study of test inclusion in agentic pull requests. We examine how often tests are included, when they are introduced during the PR lifecycle and how test-containing PRs differ from non-test PRs in terms of size, turnaround time, and merge outcomes. Across agents, test-containing PRs are more common over time and tend to be larger and take longer to complete, while merge rates remain largely similar. We also observe variation across agents in both test adoption and the balance between test and production code within test PRs. Our findings provide a descriptive view of testing behavior in agentic pull requests and offer empirical grounding for future studies of autonomous software development.
Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 3 of its 22 references that are in the map. First posted on arXiv 7 Jan 2026.
Builds on
3 of 22 references.
References · 3
Built on by
Citing papers per year: 2026: 4.
4 papers.
- 2026To What Extent Does Agent-generated Code Require Maintenance? An Empirical Studycited by 2
- 2026AIRA: AI-Induced Risk Audit: A Structured Inspection Framework for AI-Generated Codecited by 0
- 2026Do These Violent Delights Have Violent Ends? Measuring the Post-Merge Fate of Agentic Codecited by 0
- 2026Trust but Verify? Uncovering the Security Debt of Autonomous Coding Agentscited by 0
- 2026Trust but Verify? Uncovering the Security Debt of Autonomous Coding Agents14 Jul 2026
- 2026Do These Violent Delights Have Violent Ends? Measuring the Post-Merge Fate of Agentic Code10 Jul 2026
- 2026To What Extent Does Agent-generated Code Require Maintenance? An Empirical Study7 May 2026
- 2026AIRA: AI-Induced Risk Audit: A Structured Inspection Framework for AI-Generated Code19 Apr 2026
Similar, not cited
Close in wording, no citation either way
- 2026Test Coverage Analysis of Agentic Pull Requests
- 2026Human-Agent versus Human Pull Requests: A Testing-Focused Characterization and Comparison
- 2026How Do AI Coding Agents Contribute to Software Development? an Empirical Study of Agentic Pull Requests
- 2026Why Agentic-PRs Get Rejected: A Comparative Study of Coding Agents
- 2026Where Do AI Coding Agents Fail? An Empirical Study of Failed Agentic Pull Requests in GitHub
- 2026Are Coding Agents Generating Over-Mocked Tests? An Empirical Study
- 2026Beyond Bug Fixes: An Empirical Investigation of Post-Merge Code Quality Issues in Agent-Generated Pull Requests
- 2026Why Are Agentic Pull Requests Merged or Rejected? An Empirical Study