# Fine-Tuning and Prompt Engineering for Large Language Models-based Code Review Automation

arXiv: https://arxiv.org/abs/2402.00905 · HTML: https://mlsanity.com/paper/2402.00905/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Feb 2024.

## Built on by (most cited first, 23 of 23)

- [Breaking the Exploration Bottleneck: Rubric-Scaffolded Reinforcement Learning for General LLM Reasoning](https://mlsanity.com/paper/2508.16949/) (2025)
- [Human-In-the-Loop Software Development Agents](https://mlsanity.com/paper/2411.12924/) (2024)
- [The Future of AI-Driven Software Engineering](https://mlsanity.com/paper/2406.07737/) (2024)
- [CRScore: Grounding Automated Evaluation of Code Review Comments in Code Claims and Smells](https://mlsanity.com/paper/2409.19801/) (2024)
- [LAURA: Enhancing Code Review Generation with Context-Enriched Retrieval-Augmented LLM](https://mlsanity.com/paper/2512.01356/) (2025)
- [RovoDev Code Reviewer: A Large-Scale Online Evaluation of LLM-based Code Review Automation at Atlassian](https://mlsanity.com/paper/2601.01129/) (2026)
- [Automated Code Review Using Large Language Models at Ericsson: An Experience Report](https://mlsanity.com/paper/2507.19115/) (2025)
- [Understanding Dominant Themes in Reviewing Agentic AI-authored Code](https://mlsanity.com/paper/2601.19287/) (2026)
- [Code Readability in the Age of Large Language Models: An Industrial Case Study from Atlassian](https://mlsanity.com/paper/2501.11264/) (2025)
- [Supervised Fine-Tuning LLMs to Behave as Pedagogical Agents in Programming Education](https://mlsanity.com/paper/2502.20527/) (2025)
- [CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models](https://mlsanity.com/paper/2503.16167/) (2025)
- [Measuring and Exploiting Contextual Bias in LLM-Assisted Security Code Review](https://mlsanity.com/paper/2603.18740/) (2026)
- [Unlocking the Potential of Past Research: Using Generative AI to Reconstruct Healthcare Simulation Models](https://mlsanity.com/paper/2503.21646/) (2025)
- [LLM-as-a-Fuzzy-Judge: Fine-Tuning Large Language Models as a Clinical Evaluation Judge with Fuzzy Logic](https://mlsanity.com/paper/2506.11221/) (2025)
- [Refactoring with LLMs: Bridging Human Expertise and Machine Understanding](https://mlsanity.com/paper/2510.03914/) (2025)
- [A Large-Scale Empirical Analysis of Custom GPTs' Vulnerabilities in the OpenAI Ecosystem](https://mlsanity.com/paper/2505.08148/) (2025)
- [A Blueprint for AI-Driven Software Quality: Integrating LLMs with Established Standards](https://mlsanity.com/paper/2505.13766/) (2025)
- [MemoCoder: Automated Function Synthesis using LLM-Supported Agents](https://mlsanity.com/paper/2507.18812/) (2025)
- [Understanding the Limits of Automated Evaluation for Code Review Bots in Practice](https://mlsanity.com/paper/2604.24525/) (2026)
- [When Does a Second Model Help? Cross-Model Review in LLM Verification](https://mlsanity.com/paper/2610.01471/) (2026)
- [Automated Code Review Using Large Language Models with Symbolic Reasoning](https://mlsanity.com/paper/2507.18476/) (2025)
- [Enhancing Large Language Models with Retrieval Augmented Generation for Software Testing and Inspection Automation](https://mlsanity.com/paper/2604.15270/) (2026)
- [Evaluating the Impact of Explainable AI on Trust in AI-Assisted Code Review](https://mlsanity.com/paper/2607.24601/) (2026)

## Abstract (arXiv)

Context: The rapid evolution of Large Language Models (LLMs) has sparked significant interest in leveraging their capabilities for automating code review processes. Prior studies often focus on developing LLMs for code review automation, yet require expensive resources, which is infeasible for organizations with limited budgets and resources. Thus, fine-tuning and prompt engineering are the two common approaches to leveraging LLMs for code review automation. Objective: We aim to investigate the performance of LLMs-based code review automation based on two contexts, i.e., when LLMs are leveraged by fine-tuning and prompting. Fine-tuning involves training the model on a specific code review dataset, while prompting involves providing explicit instructions to guide the model's generation process without requiring a specific code review dataset. Method: We leverage model fine-tuning and inference techniques (i.e., zero-shot learning, few-shot learning and persona) on LLMs-based code review automation. In total, we investigate 12 variations of two LLMs-based code review automation (i.e., GPT- 3.5 and Magicoder), and compare them with the Guo et al.'s approach and three existing code review automation approaches. Results: The fine-tuning of GPT 3.5 with zero-shot learning helps GPT-3.5 to achieve 73.17% -74.23% higher EM than the Guo et al.'s approach. In addition, when GPT-3.5 is not fine-tuned, GPT-3.5 with few-shot learning achieves 46.38% - 659.09% higher EM than GPT-3.5 with zero-shot learning. Conclusions: Based on our results, we recommend that (1) LLMs for code review automation should be fine-tuned to achieve the highest performance; and (2) when data is not sufficient for model fine-tuning (e.g., a cold-start problem), few-shot learning without a persona should be used for LLMs for code review automation.