"Go Home Copilot, You're Drunk": Understanding Developer Responses to Agent-Generated Code Review Comments
On the map: LLM agents and tool use → Software engineering coding agents
Abstract (arXiv)
Code review is a critical quality assurance practice in software engineering development, and AI coding agents are increasingly generating review comments on pull requests. However, little is known about how developers actually respond to such agent-generated feedback. In this paper, we present the first large-scale empirical study on the resolution of agent-generated code review comments. We analyze 54,791 comments generated by five widely used coding agents (i.e., Copilot, Cursor, Codex, Devin, and Claude) across 342 Python repositories on GitHub. We examine (1) resolution rates across agents and comment types, (2) the role of developer experience, and (3) characteristics that influence comment usefulness. Our results show that resolution rate varies considerably across agents, with Copilot accounting for the majority of resolved comments (72.9%). Core developers resolve the majority of agent-generated feedback, particularly for textitdesign and textitevolvability-related comments, while peripheral developers are more involved in resolving textitfunctional defect comments. Through open card sorting of 470 unresolved comment discussions, we identify textitten discussion patterns explaining why comments remain unresolved, with textitincorrect suggestions and textitintentional design decisions being the most prevalent. Finally, our analysis reveals that the presence of an inline textitcode suggestion is the strongest predictor of comment resolution, while lengthy and complex comments are less likely to be acted upon. Our findings provide insights for improving AI-generated code review feedback and its integration into development workflows.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. First posted on arXiv 24 Jul 2026.
Builds on
No reference list yet.
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Is Agentic Code Review Helpful? Mining Developers' Feedback to CodeRabbit Reviews in the Wild
- 2026On the Footprints of Reviewer Bots Feedback on Agentic Pull Requests in OSS GitHub Repositories
- 2026More Code, Less Reuse: Investigating Code Quality and Reviewer Sentiment towards AI-generated Pull Requests
- 2026Human-AI Synergy in Agentic Code Review
- 20263100 Opinions on Code Review in an AI World: Building Causal Theory from Practitioner Discourse
- 2026Understanding Dominant Themes in Reviewing Agentic AI-authored Code
- 2026From Industry Claims to Empirical Reality: An Empirical Study of Code Review Agents in Pull Requests
- 2026From Code Review to Code Critique: Intent, Drift, and Spotlight for AI-Generated Diffs at Scale