# Using Pre-Trained Models to Boost Code Review Automation

arXiv: https://arxiv.org/abs/2201.06850 · HTML: https://mlsanity.com/paper/2201.06850/

## Key facts

- Cited by 63 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Jan 2022.

## Built on by (most cited first, 30 of 63)

- [Large Language Models for Software Engineering: A Systematic Literature Review](https://mlsanity.com/paper/2308.10620/) (2023)
- [Automating Code Review Activities by Large-Scale Pre-training](https://mlsanity.com/paper/2203.09095/) (2022)
- [LLaMA-Reviewer: Advancing Code Review Automation with Large Language Models through Parameter-Efficient Fine-Tuning](https://mlsanity.com/paper/2308.11148/) (2023)
- [CoditT5: Pretraining for Source Code and Natural Language Editing](https://mlsanity.com/paper/2208.05446/) (2022)
- [CodeScope: An Execution-based Multilingual Multitask Multidimensional Benchmark for Evaluating LLMs on Code Understanding and Generation](https://mlsanity.com/paper/2311.08588/) (2023)
- [Automatic Programming: Large Language Models and Beyond](https://mlsanity.com/paper/2405.02213/) (2024)
- [CodeAgent: Autonomous Communicative Agents for Code Review](https://mlsanity.com/paper/2402.02172/) (2024)
- [5%>100%: Breaking Performance Shackles of Full Fine-Tuning on Visual Recognition Tasks](https://mlsanity.com/paper/2408.08345/) (2024)
- [Towards Efficient Fine-tuning of Pre-trained Code Models: An Experimental Study and Beyond](https://mlsanity.com/paper/2304.05216/) (2023)
- [Adapter is All You Need for Tuning Visual Tasks](https://mlsanity.com/paper/2311.15010/) (2023)
- [AI-Assisted Assessment of Coding Practices in Modern Code Review](https://mlsanity.com/paper/2405.13565/) (2024)
- [Benchmarking Causal Study to Interpret Large Language Models for Source Code](https://mlsanity.com/paper/2308.12415/) (2023)
- [Migrating Code At Scale With LLMs At Google](https://mlsanity.com/paper/2504.09691/) (2025)
- [CodeEditor: Learning to Edit Source Code with Pre-trained Models](https://mlsanity.com/paper/2210.17040/) (2022)
- [ChatGPT: A Study on its Utility for Ubiquitous Software Engineering Tasks](https://mlsanity.com/paper/2305.16837/) (2023)
- [Software Vulnerability and Functionality Assessment using LLMs](https://mlsanity.com/paper/2403.08429/) (2024)
- [Multilingual Code Co-Evolution Using Large Language Models](https://mlsanity.com/paper/2307.14991/) (2023)
- [Automating Code-Related Tasks Through Transformers: The Impact of Pre-training](https://mlsanity.com/paper/2302.04048/) (2023)
- [CRScore: Grounding Automated Evaluation of Code Review Comments in Code Claims and Smells](https://mlsanity.com/paper/2409.19801/) (2024)
- [LAURA: Enhancing Code Review Generation with Context-Enriched Retrieval-Augmented LLM](https://mlsanity.com/paper/2512.01356/) (2025)
- [Automated User Story Generation with Test Case Specification Using Large Language Model](https://mlsanity.com/paper/2404.01558/) (2024)
- [Prompting and Fine-tuning Large Language Models for Automated Code Review Comment Generation](https://mlsanity.com/paper/2411.10129/) (2024)
- [Evaluating Large Language Models for Code Review](https://mlsanity.com/paper/2505.20206/) (2025)
- [CommitBART: A Large Pre-trained Model for GitHub Commits](https://mlsanity.com/paper/2208.08100/) (2022)
- [Combining Large Language Models with Static Analyzers for Code Review Generation](https://mlsanity.com/paper/2502.06633/) (2025)
- [Retrieval-Augmented Code Review Comment Generation](https://mlsanity.com/paper/2506.11591/) (2025)
- [RovoDev Code Reviewer: A Large-Scale Online Evaluation of LLM-based Code Review Automation at Atlassian](https://mlsanity.com/paper/2601.01129/) (2026)
- [SecureReviewer: Enhancing Large Language Models for Secure Code Review through Secure-aware Fine-tuning](https://mlsanity.com/paper/2510.26457/) (2025)
- [HalluJudge: A Reference-Free Hallucination Detection for Context Misalignment in Code Review Automation](https://mlsanity.com/paper/2601.19072/) (2026)
- [Understanding Dominant Themes in Reviewing Agentic AI-authored Code](https://mlsanity.com/paper/2601.19287/) (2026)

## Abstract (arXiv)

Code review is a practice widely adopted in open source and industrial projects. Given the non-negligible cost of such a process, researchers started investigating the possibility of automating specific code review tasks. We recently proposed Deep Learning (DL) models targeting the automation of two tasks: the first model takes as input a code submitted for review and implements in it changes likely to be recommended by a reviewer; the second takes as input the submitted code and a reviewer comment posted in natural language and automatically implements the change required by the reviewer. While the preliminary results we achieved are encouraging, both models had been tested in rather simple code review scenarios, substantially simplifying the targeted problem. This was also due to the choices we made when designing both the technique and the experiments. In this paper, we build on top of that work by demonstrating that a pre-trained Text-To-Text Transfer Transformer (T5) model can outperform previous DL models for automating code review tasks. Also, we conducted our experiments on a larger and more realistic (and challenging) dataset of code review activities.