# BitsAI-CR: Automated Code Review via LLM in Practice

arXiv: https://arxiv.org/abs/2501.15134 · HTML: https://mlsanity.com/paper/2501.15134/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Jan 2025.

## Built on by (most cited first, 22 of 22)

- [P2P: Automated Paper-to-Poster Generation and Fine-Grained Benchmark](https://mlsanity.com/paper/2505.17104/) (2025)
- [Uncovering Vulnerabilities of LLM-Assisted Cyber Threat Intelligence](https://mlsanity.com/paper/2509.23573/) (2025)
- [AACR-Bench: Evaluating Automatic Code Review with Holistic Repository-Level Context](https://mlsanity.com/paper/2601.19494/) (2026)
- [SWE-Review: Closing the Loop on Issue Resolution with Agentic Code Review](https://mlsanity.com/paper/2607.06065/) (2026)
- [Issue-Oriented Agent-Based Framework for Automated Review Comment Generation](https://mlsanity.com/paper/2511.00517/) (2025)
- [Sphinx: Benchmarking and Modeling for LLM-Driven Pull Request Review](https://mlsanity.com/paper/2601.04252/) (2026)
- [CTFExplorer: Evaluating LLM Offensive Agents Through Multi-Target Web CTF Benchmarking](https://mlsanity.com/paper/2602.08023/) (2026)
- [Measuring and Exploiting Contextual Bias in LLM-Assisted Security Code Review](https://mlsanity.com/paper/2603.18740/) (2026)
- [AgenticSCR: An Autonomous Agentic Secure Code Review for Immature Vulnerabilities Detection](https://mlsanity.com/paper/2601.19138/) (2026)
- [BitsAI-Fix: LLM-Driven Approach for Automated Lint Error Resolution in Practice](https://mlsanity.com/paper/2508.03487/) (2025)
- [Fine-Tuning LLMs to Analyze Multiple Dimensions of Code Review: A Maximum Entropy Regulated Long Chain-of-Thought Approach](https://mlsanity.com/paper/2509.21170/) (2025)
- [Tracing Stereotypes in Pre-trained Transformers: From Biased Neurons to Fairer Models](https://mlsanity.com/paper/2601.05663/) (2026)
- [Understanding the Limits of Automated Evaluation for Code Review Bots in Practice](https://mlsanity.com/paper/2604.24525/) (2026)
- [Rethinking Code Review in the Age of AI: A Vision for Agentic Code Review](https://mlsanity.com/paper/2605.17548/) (2026)
- [SEVRA-BENCH: Social Engineering of Vulnerabilities in Review Agents](https://mlsanity.com/paper/2606.13757/) (2026)
- [ConceptRM: The Quest to Mitigate Alert Fatigue through Consensus-Based Purity-Driven Data Cleaning for Reflection Modelling](https://mlsanity.com/paper/2602.20166/) (2026)
- [Learning From Developers: Towards Reliable Patch Validation at Scale for Linux](https://mlsanity.com/paper/2603.24825/) (2026)
- [Archer: Towards Agentic Review for Compiler Optimizations](https://mlsanity.com/paper/2607.01808/) (2026)
- [Evaluating the Impact of Explainable AI on Trust in AI-Assisted Code Review](https://mlsanity.com/paper/2607.24601/) (2026)
- [Evaluation Pitfalls and Sparsity Limitations in LLM-based Confidence Estimates for Classification](https://mlsanity.com/paper/2608.04899/) (2026)
- [From Static to Dynamic: Benchmarking Real-World Code Review with MCR-Bench](https://mlsanity.com/paper/2608.27442/) (2026)
- [Using Agentic AI for contextualized and multifaceted code review at Ericsson](https://mlsanity.com/paper/2609.15877/) (2026)

## Abstract (arXiv)

Code review remains a critical yet resource-intensive process in software development, particularly challenging in large-scale industrial environments. While Large Language Models (LLMs) show promise for automating code review, existing solutions face significant limitations in precision and practicality. This paper presents BitsAI-CR, an innovative framework that enhances code review through a two-stage approach combining RuleChecker for initial issue detection and ReviewFilter for precision verification. The system is built upon a comprehensive taxonomy of review rules and implements a data flywheel mechanism that enables continuous performance improvement through structured feedback and evaluation metrics. Our approach introduces an Outdated Rate metric that can reflect developers' actual adoption of review comments, enabling automated evaluation and systematic optimization at scale. Empirical evaluation demonstrates BitsAI-CR's effectiveness, achieving 75.0% precision in review comment generation. For the Go language which has predominant usage at ByteDance, we maintain an Outdated Rate of 26.7%. The system has been successfully deployed at ByteDance, serving over 12,000 Weekly Active Users (WAU). Our work provides valuable insights into the practical application of automated code review and offers a blueprint for organizations seeking to implement automated code reviews at scale.