# Improving Automated Code Reviews: Learning from Experience

arXiv: https://arxiv.org/abs/2402.03777 · HTML: https://mlsanity.com/paper/2402.03777/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Feb 2024.

## Built on by (most cited first, 13 of 13)

- [CRScore: Grounding Automated Evaluation of Code Review Comments in Code Claims and Smells](https://mlsanity.com/paper/2409.19801/) (2024)
- [HalluJudge: A Reference-Free Hallucination Detection for Context Misalignment in Code Review Automation](https://mlsanity.com/paper/2601.19072/) (2026)
- [Leveraging Reviewer Experience in Code Review Comment Generation](https://mlsanity.com/paper/2409.10959/) (2024)
- [Hallucinations in Code Change to Natural Language Generation: Prevalence and Evaluation of Detection Metrics](https://mlsanity.com/paper/2508.08661/) (2025)
- [CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models](https://mlsanity.com/paper/2503.16167/) (2025)
- [Sphinx: Benchmarking and Modeling for LLM-Driven Pull Request Review](https://mlsanity.com/paper/2601.04252/) (2026)
- [What Challenges Do Developers Face in AI Agent Systems? An Empirical Study on Stack Overflow & GitHub Issues](https://mlsanity.com/paper/2510.25423/) (2025)
- [Is Agentic Code Review Helpful? Mining Developers' Feedback to CodeRabbit Reviews in the Wild](https://mlsanity.com/paper/2607.03316/) (2026)
- [Improving Automated Secure Code Reviews: A Synthetic Dataset for Code Vulnerability Flaws](https://mlsanity.com/paper/2504.16310/) (2025)
- [Fine-Tuning Multilingual Language Models for Code Review: An Empirical Study on Industrial C# Projects](https://mlsanity.com/paper/2507.19271/) (2025)
- [Exploring the Potential of Large Language Models in Fine-Grained Review Comment Classification](https://mlsanity.com/paper/2508.09832/) (2025)
- [From Pilots to Practices: A Scoping Review of GenAI-Enabled Personalization in Computer Science Education](https://mlsanity.com/paper/2512.20714/) (2025)
- [Bespoke Visual Assistance: What and How do Blind and Low-Vision People Create with Agentic Programming?](https://mlsanity.com/paper/2607.21760/) (2026)

## Abstract (arXiv)

Modern code review is a critical quality assurance process that is widely adopted in both industry and open source software environments. This process can help newcomers learn from the feedback of experienced reviewers; however, it often brings a large workload and stress to reviewers. To alleviate this burden, the field of automated code reviews aims to automate the process, teaching large language models to provide reviews on submitted code, just as a human would. A recent approach pre-trained and fine-tuned the code intelligent language model on a large-scale code review corpus. However, such techniques did not fully utilise quality reviews amongst the training data. Indeed, reviewers with a higher level of experience or familiarity with the code will likely provide deeper insights than the others. In this study, we set out to investigate whether higher-quality reviews can be generated from automated code review models that are trained based on an experience-aware oversampling technique. Through our quantitative and qualitative evaluation, we find that experience-aware oversampling can increase the correctness, level of information, and meaningfulness of reviews generated by the current state-of-the-art model without introducing new data. The results suggest that a vast amount of high-quality reviews are underutilised with current training strategies. This work sheds light on resource-efficient ways to boost automated code review models.