# MESIA: Understanding and Leveraging Supplementary Nature of Method-level Comments for Automatic Comment Generation

arXiv: https://arxiv.org/abs/2403.17357 · HTML: https://mlsanity.com/paper/2403.17357/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 59 references that are in the map.
- First posted on arXiv 26 Mar 2024.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation](https://mlsanity.com/paper/2109.00859/) (2021)
- [code2seq: Generating Sequences from Structured Representations of Code](https://mlsanity.com/paper/1808.01400/) (2018)
- [A Transformer-based Approach for Source Code Summarization](https://mlsanity.com/paper/2005.00653/) (2020)
- [Few-shot training LLMs for project-specific code-summarization](https://mlsanity.com/paper/2207.04237/) (2022)
- [Human-like Summarization Evaluation with ChatGPT](https://mlsanity.com/paper/2304.02554/) (2023)
- [Improving Automatic Source Code Summarization via Deep Reinforcement Learning](https://mlsanity.com/paper/1811.07234/) (2018)
- [A Neural Model for Generating Natural Language Summaries of Program Subroutines](https://mlsanity.com/paper/1902.01954/) (2019)
- [Large Language Models are Few-Shot Summarizers: Multi-Intent Comment Generation via In-Context Learning](https://mlsanity.com/paper/2304.11384/) (2023)
- [Studying the Usage of Text-To-Text Transfer Transformer to Support Code-Related Tasks](https://mlsanity.com/paper/2102.02017/) (2021)
- [On the Evaluation of Neural Code Summarization](https://mlsanity.com/paper/2107.07112/) (2021)
- [Automatic Code Documentation Generation Using GPT-3](https://mlsanity.com/paper/2209.02235/) (2022)
- [Retrieve and Refine: Exemplar-based Neural Comment Generation](https://mlsanity.com/paper/1910.10419/) (2019)
- [Automatic Code Summarization via ChatGPT: How Far Are We?](https://mlsanity.com/paper/2305.12865/) (2023)
- [Code to Comment "Translation": Data, Metrics, Baselining & Evaluation](https://mlsanity.com/paper/2010.01410/) (2020)
- [Are We Building on the Rock? On the Importance of Data Preprocessing for Code Summarization](https://mlsanity.com/paper/2207.05579/) (2022)
- [EditSum: A Retrieve-and-Edit Framework for Source Code Summarization](https://mlsanity.com/paper/2308.13775/) (2023)
- [Developer-Intent Driven Code Comment Generation](https://mlsanity.com/paper/2302.07055/) (2023)
- [A Survey of Automatic Generation of Source Code Comments: Algorithms and Techniques](https://mlsanity.com/paper/1907.10863/) (2019)
- [AST-Transformer: Encoding Abstract Syntax Trees Efficiently for Code Summarization](https://mlsanity.com/paper/2112.01184/) (2021)
- [A Decade of Code Comment Quality Assessment: A Systematic Literature Review](https://mlsanity.com/paper/2209.08165/) (2022)
- [Speculative Analysis for Quality Assessment of Code Comments](https://mlsanity.com/paper/2102.09605/) (2021)

## Built on by (most cited first, 2 of 2)

- [From Code Foundation Models to Agents and Applications: A Comprehensive Survey and Practical Guide to Code Intelligence](https://mlsanity.com/paper/2511.18538/) (2025)
- [Issue Retrieval and Verification Enhanced Supplementary Code Comment Generation](https://mlsanity.com/paper/2506.14649/) (2025)

## Abstract (arXiv)

Code comments are important for developers in program comprehension. In scenarios of comprehending and reusing a method, developers expect code comments to provide supplementary information beyond the method signature. However, the extent of such supplementary information varies a lot in different code comments. In this paper, we raise the awareness of the supplementary nature of method-level comments and propose a new metric named MESIA (Mean Supplementary Information Amount) to assess the extent of supplementary information that a code comment can provide. With the MESIA metric, we conduct experiments on a popular code-comment dataset and three common types of neural approaches to generate method-level comments. Our experimental results demonstrate the value of our proposed work with a number of findings. (1) Small-MESIA comments occupy around 20% of the dataset and mostly fall into only the WHAT comment category. (2) Being able to provide various kinds of essential information, large-MESIA comments in the dataset are difficult for existing neural approaches to generate. (3) We can improve the capability of existing neural approaches to generate large-MESIA comments by reducing the proportion of small-MESIA comments in the training set. (4) The retrained model can generate large-MESIA comments that convey essential meaningful supplementary information for methods in the small-MESIA test set, but will get a lower BLEU score in evaluation. These findings indicate that with good training data, auto-generated comments can sometimes even surpass human-written reference comments, and having no appropriate ground truth for evaluation is an issue that needs to be addressed by future work on automatic comment generation.