# AI-Mediated Code Comment Improvement

arXiv: https://arxiv.org/abs/2505.09021 · HTML: https://mlsanity.com/paper/2505.09021/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 66 references that are in the map.
- First posted on arXiv 13 May 2025.

## Builds on (most cited first, 23 of 23)

- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [CodeBERT: A Pre-Trained Model for Programming and Natural Languages](https://mlsanity.com/paper/2002.08155/) (2020)
- [A Transformer-based Approach for Source Code Summarization](https://mlsanity.com/paper/2005.00653/) (2020)
- [The Adverse Effects of Code Duplication in Machine Learning Models of Code](https://mlsanity.com/paper/1812.06469/) (2018)
- [Improved Code Summarization via a Graph Neural Network](https://mlsanity.com/paper/2004.02843/) (2020)
- [A Neural Model for Generating Natural Language Summaries of Program Subroutines](https://mlsanity.com/paper/1902.01954/) (2019)
- [Large Language Models are Few-Shot Summarizers: Multi-Intent Comment Generation via In-Context Learning](https://mlsanity.com/paper/2304.11384/) (2023)
- [Can LLMs Replace Manual Annotation of Software Engineering Artifacts?](https://mlsanity.com/paper/2408.05534/) (2024)
- [Retrieve and Refine: Exemplar-based Neural Comment Generation](https://mlsanity.com/paper/1910.10419/) (2019)
- [Distilled GPT for Source Code Summarization](https://mlsanity.com/paper/2308.14731/) (2023)
- [Project-Level Encoding for Neural Source Code Summarization of Subroutines](https://mlsanity.com/paper/2103.11599/) (2021)
- [Are We Building on the Rock? On the Importance of Data Preprocessing for Code Summarization](https://mlsanity.com/paper/2207.05579/) (2022)
- [Developer-Intent Driven Code Comment Generation](https://mlsanity.com/paper/2302.07055/) (2023)
- [A Survey of Automatic Generation of Source Code Comments: Algorithms and Techniques](https://mlsanity.com/paper/1907.10863/) (2019)
- [Socio-Technical Grounded Theory for Software Engineering](https://mlsanity.com/paper/2103.14235/) (2021)
- [A Language Model of Java Methods with Train/Test Deduplication](https://mlsanity.com/paper/2305.08286/) (2023)
- [Prompting and Fine-tuning Large Language Models for Automated Code Review Comment Generation](https://mlsanity.com/paper/2411.10129/) (2024)
- [Semantic Similarity Loss for Neural Source Code Summarization](https://mlsanity.com/paper/2308.07429/) (2023)
- [Autonomy Is An Acquired Taste: Exploring Developer Preferences for GitHub Bots](https://mlsanity.com/paper/2302.05048/) (2023)
- [Do you really code? Designing and Evaluating Screening Questions for Online Surveys with Programmers](https://mlsanity.com/paper/2103.04429/) (2021)
- [Automatic Comment Generation via Multi-Pass Deliberation](https://mlsanity.com/paper/2209.06634/) (2022)
- [Statement-based Memory for Neural Source Code Summarization](https://mlsanity.com/paper/2307.11709/) (2023)

## Abstract (arXiv)

This paper describes an approach to improve code comments along different quality axes by rewriting those comments with customized Artificial Intelligence (AI)-based tools. We conduct an empirical study followed by grounded theory qualitative analysis to determine the quality axes to improve. Then we propose a procedure using a Large Language Model (LLM) to rewrite existing code comments along the quality axes. We implement our procedure using GPT-4o, then distil the results into a smaller model capable of being run in-house, so users can maintain data custody. We evaluate both our approach using GPT-4o and the distilled model versions. We show in an evaluation how our procedure improves code comments along the quality axes. We release all data and source code in an online repository for reproducibility.