# Predicting Usefulness of Code Review Comments using Textual Features and Developer Experience

arXiv: https://arxiv.org/abs/1807.04485 · HTML: https://mlsanity.com/paper/1807.04485/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Jul 2018.

## Built on by (most cited first, 13 of 13)

- [CRScore: Grounding Automated Evaluation of Code Review Comments in Code Claims and Smells](https://mlsanity.com/paper/2409.19801/) (2024)
- [The Forgotten Role of Search Queries in IR-based Bug Localization: An Empirical Study](https://mlsanity.com/paper/2108.05341/) (2021)
- [Leveraging Reviewer Experience in Code Review Comment Generation](https://mlsanity.com/paper/2409.10959/) (2024)
- [CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models](https://mlsanity.com/paper/2503.16167/) (2025)
- [Generative AI for Software Metadata: Overview of the Information Retrieval in Software Engineering Track at FIRE 2023](https://mlsanity.com/paper/2311.03374/) (2023)
- [On Unified Prompt Tuning for Request Quality Assurance in Public Code Review](https://mlsanity.com/paper/2404.07942/) (2024)
- [Rethinking Code Review in the Age of AI: A Vision for Agentic Code Review](https://mlsanity.com/paper/2605.17548/) (2026)
- [Is Agentic Code Review Helpful? Mining Developers' Feedback to CodeRabbit Reviews in the Wild](https://mlsanity.com/paper/2607.03316/) (2026)
- [A study of the impact of generative AI-based data augmentation on software metadata classification](https://mlsanity.com/paper/2310.13714/) (2023)
- [Leveraging Generative AI: Improving Software Metadata Classification with Generated Code-Comment Pairs](https://mlsanity.com/paper/2311.03365/) (2023)
- [Knowledge-Guided Prompt Learning for Request Quality Assurance in Public Code Review](https://mlsanity.com/paper/2410.21673/) (2024)
- [Exploring the Potential of Large Language Models in Fine-Grained Review Comment Classification](https://mlsanity.com/paper/2508.09832/) (2025)
- [LLMs for Qualitative Data Analysis Fail on Security-specificComments in Human Experiments](https://mlsanity.com/paper/2604.10834/) (2026)

## Abstract (arXiv)

Although peer code review is widely adopted in both commercial and open source development, existing studies suggest that such code reviews often contain a significant amount of non-useful review comments. Unfortunately, to date, no tools or techniques exist that can provide automatic support in improving those non-useful comments. In this paper, we first report a comparative study between useful and non-useful review comments where we contrast between them using their textual characteristics, and reviewers' experience. Then, based on the findings from the study, we develop RevHelper, a prediction model that can help the developers improve their code review comments through automatic prediction of their usefulness during review submission. Comparative study using 1,116 review comments suggested that useful comments share more vocabulary with the changed code, contain salient items like relevant code elements, and their reviewers are generally more experienced. Experiments using 1,482 review comments report that our model can predict comment usefulness with 66% prediction accuracy which is promising. Comparison with three variants of a baseline model using a case study validates our empirical findings and demonstrates the potential of our model.