# Rethinking Code Review Workflows with LLM Assistance: An Empirical Study

arXiv: https://arxiv.org/abs/2505.16339 · HTML: https://mlsanity.com/paper/2505.16339/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 May 2025.

## Built on by (most cited first, 10 of 10)

- [Hallucinations in Code Change to Natural Language Generation: Prevalence and Evaluation of Detection Metrics](https://mlsanity.com/paper/2508.08661/) (2025)
- [Trust Me, I Know This Function: Hijacking LLM Static Analysis using Bias](https://mlsanity.com/paper/2508.17361/) (2025)
- [Measuring and Exploiting Contextual Bias in LLM-Assisted Security Code Review](https://mlsanity.com/paper/2603.18740/) (2026)
- [When AI Reviews Its Own Code: Recursive Self-Training Collapse in Code LLMs](https://mlsanity.com/paper/2606.28438/) (2026)
- [Rethinking Code Review in the Age of AI: A Vision for Agentic Code Review](https://mlsanity.com/paper/2605.17548/) (2026)
- [Agentic Code Review in the Terminal: A Trajectory-Level Analysis of Behavior, Cost, and Human-Alignment](https://mlsanity.com/paper/2607.16740/) (2026)
- [Archer: Towards Agentic Review for Compiler Optimizations](https://mlsanity.com/paper/2607.01808/) (2026)
- [From Human-Centric to Agentic Code Review: The Impact of Different Generations of Generative AI Technology on Review Quality](https://mlsanity.com/paper/2607.13196/) (2026)
- [Evaluating the Impact of Explainable AI on Trust in AI-Assisted Code Review](https://mlsanity.com/paper/2607.24601/) (2026)
- [Cross-Provider Review as a Runtime Contract for Coding Agents: A Controlled Pilot and Fault-Injection Study](https://mlsanity.com/paper/2610.10961/) (2026)

## Abstract (arXiv)

Code reviews are a critical yet time-consuming aspect of modern software development, increasingly challenged by growing system complexity and the demand for faster delivery. This paper presents a study conducted at WirelessCar Sweden AB, combining an exploratory field study of current code review practices with a field experiment involving two variations of an LLM-assisted code review tool. The field study identifies key challenges in traditional code reviews, including frequent context switching, insufficient contextual information, and highlights both opportunities (e.g., automatic summarization of complex pull requests) and concerns (e.g., false positives and trust issues) in using LLMs. In the field experiment, we developed two prototype variations: one offering LLM-generated reviews upfront and the other enabling on-demand interaction. Both utilize a semantic search pipeline based on retrieval-augmented generation to assemble relevant contextual information for the review, thereby tackling the uncovered challenges. Developers evaluated both variations in real-world settings: AI-led reviews are overall more preferred, while still being conditional on the reviewers' familiarity with the code base, as well as on the severity of the pull request.