# Understanding Dominant Themes in Reviewing Agentic AI-authored Code

arXiv: https://arxiv.org/abs/2601.19287 · HTML: https://mlsanity.com/paper/2601.19287/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use.
- Builds on 8 of its 21 references that are in the map.
- First posted on arXiv 27 Jan 2026.

## Placed in

- LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/)

## Builds on (most cited first, 8 of 8)

- [Automating Code Review Activities by Large-Scale Pre-training](https://mlsanity.com/paper/2203.09095/) (2022)
- [Using Pre-Trained Models to Boost Code Review Automation](https://mlsanity.com/paper/2201.06850/) (2022)
- [On the Use of Agentic Coding: An Empirical Study of Pull Requests on GitHub](https://mlsanity.com/paper/2509.14745/) (2025)
- [AUGER: Automatically Generating Review Comments with Pre-training Models](https://mlsanity.com/paper/2208.08014/) (2022)
- [Fine-Tuning and Prompt Engineering for Large Language Models-based Code Review Automation](https://mlsanity.com/paper/2402.00905/) (2024)
- [AI-powered Code Review with LLMs: Early Results](https://mlsanity.com/paper/2404.18496/) (2024)
- [AI-Assisted Assessment of Coding Practices in Modern Code Review](https://mlsanity.com/paper/2405.13565/) (2024)
- [Prompting and Fine-tuning Large Language Models for Automated Code Review Comment Generation](https://mlsanity.com/paper/2411.10129/) (2024)

## Built on by (most cited first, 5 of 5)

- [3100 Opinions on Code Review in an AI World: Building Causal Theory from Practitioner Discourse](https://mlsanity.com/paper/2607.07980/) (2026)
- [Do These Violent Delights Have Violent Ends? Measuring the Post-Merge Fate of Agentic Code](https://mlsanity.com/paper/2607.09902/) (2026)
- [Trust but Verify? Uncovering the Security Debt of Autonomous Coding Agents](https://mlsanity.com/paper/2607.12428/) (2026)
- [Bespoke Visual Assistance: What and How do Blind and Low-Vision People Create with Agentic Programming?](https://mlsanity.com/paper/2607.21760/) (2026)
- [Who Finishes the Job? A Study of Follow-Up Fixes and Commit Authorship on AI Coding Agent Pull Requests](https://mlsanity.com/paper/2609.26847/) (2026)

## Abstract (arXiv)

While prior work has examined the generation capabilities of Agentic AI systems, little is known about how reviewers respond to AI-authored code in practice. In this paper, we present a large-scale empirical study of code review dynamics in agent-generated PRs. Using a curated subset of the AIDev dataset, we analyze 19,450 inline review comments spanning 3,177 agent-authored PRs from real-world GitHub repositories. We first derive a taxonomy of 12 review comment themes using topic modeling combined with large language model (LLM)-assisted semantic clustering and consolidation. According to this taxonomy, we then investigate whether zero-shot prompts to LLM can reliably annotate review comments. Our evaluation against human annotations shows that open-source LLM achieves reasonably high exact match (78.63%), macro F1 score (0.78), and substantial agreement with human annotators at the review comment level. At the PR level, the LLM also correctly identifies the dominant review theme with 78% Top-1 accuracy and achieves an average Jaccard similarity of 0.76, indicating strong alignment with human judgments. Applying this annotation pipeline at scale, we find that apart from functional correctness and logical changes, reviews of agent-authored PRs predominantly focus on documentation gaps, refactoring needs, styling and formatting issues, with testing and security-related concerns. These findings suggest that while AI agents can accelerate code production, there remain gaps requiring targeted human review oversight.