# On the Footprints of Reviewer Bots Feedback on Agentic Pull Requests in OSS GitHub Repositories

arXiv: https://arxiv.org/abs/2604.24450 · HTML: https://mlsanity.com/paper/2604.24450/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use.
- Builds on 11 of its 26 references that are in the map.
- First posted on arXiv 27 Apr 2026.

## Placed in

- LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/)

## Builds on (most cited first, 11 of 11)

- [An Empirical Evaluation of Using Large Language Models for Automated Unit Test Generation](https://mlsanity.com/paper/2302.06527/) (2023)
- [Automating Code Review Activities by Large-Scale Pre-training](https://mlsanity.com/paper/2203.09095/) (2022)
- [On the Use of Agentic Coding: An Empirical Study of Pull Requests on GitHub](https://mlsanity.com/paper/2509.14745/) (2025)
- [A ground-truth dataset and classification model for detecting bots in GitHub issue and PR comments](https://mlsanity.com/paper/2010.03303/) (2020)
- [Quality Gatekeepers: Investigating the Effects ofCode Review Bots on Pull Request Activities](https://mlsanity.com/paper/2103.13547/) (2021)
- [Evaluating Large Language Models for Code Review](https://mlsanity.com/paper/2505.20206/) (2025)
- [Does AI Code Review Lead to Code Changes? A Case Study of GitHub Actions](https://mlsanity.com/paper/2508.18771/) (2025)
- [Harnessing Large Language Models for Curated Code Reviews](https://mlsanity.com/paper/2502.03425/) (2025)
- [Suggestion Bot: Analyzing the Impact of Automated Suggested Changes on Code Reviews](https://mlsanity.com/paper/2305.06328/) (2023)
- [Explaining Explanation: An Empirical Study on Explanation in Code Reviews](https://mlsanity.com/paper/2311.09020/) (2023)
- [The Impact of Large Language Models (LLMs) on Code Review Process](https://mlsanity.com/paper/2508.11034/) (2025)

## Built on by (most cited first, 2 of 2)

- [Where Accountability Lives: Mapping Human Responsibility to Workflow Artifacts in Agentic Software Development](https://mlsanity.com/paper/2608.15678/) (2026)
- [3100 Opinions on Code Review in an AI World: Building Causal Theory from Practitioner Discourse](https://mlsanity.com/paper/2607.07980/) (2026)

## Abstract (arXiv)

Autonomous coding agents are reshaping software development by creating pull requests (PRs) on GitHub, referred to as agentic PRs. In parallel, the review process is also becoming autonomous, thereby making reviewer bots key actors in the assessment of these agentic PRs. However, their influence on PR acceptance and resolution remains unclear. This study empirically investigates the relationship between reviewer-bot feedback and PR outcomes by analyzing how Reviewer Bot Feedback Quality (relevance, clarity, conciseness) and Reviewer Bot Activity Volume (comment count) are associated with PR acceptance and resolution time. We analyze 7,416 reviewer-bot comments on 4,532 PRs from the AI_Dev dataset (a dataset that captured AI agents' PRs in GitHub projects). Our results show that reviewer-bot comments mainly focus on bug fixes, testing, and documentation, are civil in tone, and are prescriptive in nature. Reviewer bots generally produce clear and concise feedback, though the semantic relevance of comments to underlying code changes is moderate. We find that higher Reviewer Bot Activity volume is associated with longer PR resolution times and lower average feedback quality, showing that as bots generate more comments on a PR, the average pertinence of that feedback appears to degrade. At the same time, Reviewer Bot Feedback Quality shows no meaningful association with workflow outcomes. Our findings suggest that, in agentic PR workflows, reviewer bots should prioritize targeted high-relevance feedback over generating large numbers of comments.