Human-Agent versus Human Pull Requests: A Testing-Focused Characterization and Comparison
On the map: LLM agents and tool use → Software engineering coding agents
Abstract (arXiv)
AI-based coding agents are increasingly integrated into software development workflows, collaborating with developers to create pull requests (PRs). Despite their growing adoption, the role of human-agent collaboration in software testing remains poorly understood. This paper presents an empirical study of 6,582 human-agent PRs (HAPRs) and 3,122 human PRs (HPRs) from the AIDev dataset. We compare HAPRs and HPRs along three dimensions: (i) testing frequency and extent, (ii) types of testing-related changes (code-and-test co-evolution vs. test-focused), and (iii) testing quality, measured by test smells. Our findings reveal that, although the likelihood of including tests is comparable (42.9% for HAPRs vs. 40.0% for HPRs), HAPRs exhibit a larger extent of testing, nearly doubling the test-to-source line ratio found in HPRs. While test-focused task distributions are comparable, HAPRs are more likely to add new tests during co-evolution (OR=1.79), whereas HPRs prioritize modifying existing tests. Finally, although some test smell categories differ statistically, negligible effect sizes suggest no meaningful differences in quality. These insights provide the first characterization of how human-agent collaboration shapes testing practices.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. First posted on arXiv 29 Jan 2026.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026Do Autonomous Agents Contribute Test Code? A Study of Tests in Agentic Pull Requests
- 2026Humans Integrate, Agents Fix: How Agent-Authored Pull Requests Are Referenced in Practice
- 2026Test Coverage Analysis of Agentic Pull Requests
- 2026How Do AI Coding Agents Contribute to Software Development? an Empirical Study of Agentic Pull Requests
- 2026Code Change Characteristics and Description Alignment: A Comparative Study of Agentic versus Human Pull Requests
- 2026Human-AI Synergy in Agentic Code Review
- 2026Let's Make Every Pull Request Meaningful: An Empirical Analysis of Developer and Agentic Pull Requests
- 2026Safer Builders, Risky Maintainers: A Comparative Study of Breaking Changes in Human vs Agentic PRs