# Quality Gatekeepers: Investigating the Effects ofCode Review Bots on Pull Request Activities

arXiv: https://arxiv.org/abs/2103.13547 · HTML: https://mlsanity.com/paper/2103.13547/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Mar 2021.

## Built on by (most cited first, 8 of 8)

- [Motivations, Challenges, Best Practices, and Benefits for Bots and Conversational Agents in Software Engineering: A Multivocal Literature Review](https://mlsanity.com/paper/2409.11864/) (2024)
- [Predicting the First Response Latency of Maintainers and Contributors in Pull Requests](https://mlsanity.com/paper/2311.07786/) (2023)
- [Understanding the Helpfulness of Stale Bot for Pull-based Development: An Empirical Study of 20 Large Open-Source Projects](https://mlsanity.com/paper/2305.18150/) (2023)
- [On the Footprints of Reviewer Bots Feedback on Agentic Pull Requests in OSS GitHub Repositories](https://mlsanity.com/paper/2604.24450/) (2026)
- [Beyond Simpson's Paradox: A Cascade of Confounders in AI Agent Pull-Request Co-Authorship](https://mlsanity.com/paper/2606.22711/) (2026)
- [Where Accountability Lives: Mapping Human Responsibility to Workflow Artifacts in Agentic Software Development](https://mlsanity.com/paper/2608.15678/) (2026)
- [A Human Behavioral Baseline for Collective Governance in Software Projects](https://mlsanity.com/paper/2510.08956/) (2025)
- [Govern the Repository, Not the Agent: Measuring Ecosystem-Level Risk in AI-Native Software](https://mlsanity.com/paper/2606.28235/) (2026)

## Abstract (arXiv)

Software bots have been facilitating several development activities in Open Source Software (OSS) projects, including code review. However, these bots may bring unexpected impacts to group dynamics, as frequently occurs with new technology adoption. Understanding and anticipating such effects is important for planning and management. To analyze these effects, we investigate how several activity indicators change after the adoption of a code review bot. We employed a regression discontinuity design on 1,194 software projects from GitHub. We also interviewed 12 practitioners, including open-source maintainers and contributors. Our results indicate that the adoption of code review bots increases the number of monthly merged pull requests, decreases monthly non-merged pull requests, and decreases communication among developers. From the developers' perspective, these effects are explained by the transparency and confidence the bot comments introduce, in addition to the changes in the discussion focused on pull requests. Practitioners and maintainers may leverage our results to understand, or even predict, bot effects on their projects.