# Improving Automatic Source Code Summarization via Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/1811.07234 · HTML: https://mlsanity.com/paper/1811.07234/

## Key facts

- Cited by 100 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Nov 2018.

## Built on by (most cited first, 30 of 100)

- [ChatDev: Communicative Agents for Software Development](https://mlsanity.com/paper/2307.07924/) (2023)
- [CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation](https://mlsanity.com/paper/2102.04664/) (2021)
- [Large Language Models for Software Engineering: A Systematic Literature Review](https://mlsanity.com/paper/2308.10620/) (2023)
- [Large Language Models Meet NL2Code: A Survey](https://mlsanity.com/paper/2212.09420/) (2022)
- [Retrieval-Augmented Generation for Code Summarization via Hybrid GNN](https://mlsanity.com/paper/2006.05405/) (2020)
- [Is ChatGPT the Ultimate Programming Assistant -- How far is it?](https://mlsanity.com/paper/2304.11938/) (2023)
- [A Systematic Survey of Text Summarization: From Statistical Methods to Large Language Models](https://mlsanity.com/paper/2406.11289/) (2024)
- [What Do They Capture? -- A Structural Analysis of Pre-Trained Language Models for Source Code](https://mlsanity.com/paper/2202.06840/) (2022)
- [On the Evaluation of Neural Code Summarization](https://mlsanity.com/paper/2107.07112/) (2021)
- [Compilable Neural Code Generation with Compiler Feedback](https://mlsanity.com/paper/2203.05132/) (2022)
- [WebCode2M: A Real-World Dataset for Code Generation from Webpage Designs](https://mlsanity.com/paper/2404.06369/) (2024)
- [Multilingual training for Software Engineering](https://mlsanity.com/paper/2112.02043/) (2021)
- [No More Fine-Tuning? An Experimental Evaluation of Prompt Tuning in Code Intelligence](https://mlsanity.com/paper/2207.11680/) (2022)
- [Automatic Code Summarization via ChatGPT: How Far Are We?](https://mlsanity.com/paper/2305.12865/) (2023)
- [Distilled GPT for Source Code Summarization](https://mlsanity.com/paper/2308.14731/) (2023)
- [Code Structure Guided Transformer for Source Code Summarization](https://mlsanity.com/paper/2104.09340/) (2021)
- [CodeQA: A Question Answering Dataset for Source Code Comprehension](https://mlsanity.com/paper/2109.08365/) (2021)
- [On the Transferability of Pre-trained Language Models for Low-Resource Programming Languages](https://mlsanity.com/paper/2204.09653/) (2022)
- [Project-Level Encoding for Neural Source Code Summarization of Subroutines](https://mlsanity.com/paper/2103.11599/) (2021)
- [A Survey on Machine Learning Techniques for Source Code Analysis](https://mlsanity.com/paper/2110.09610/) (2021)
- [CodeIP: A Grammar-Guided Multi-Bit Watermark for Large Language Models of Code](https://mlsanity.com/paper/2404.15639/) (2024)
- [Deep Learning for Code Intelligence: Survey, Benchmark and Toolkit](https://mlsanity.com/paper/2401.00288/) (2023)
- [Improving Code Summarization with Block-wise Abstract Syntax Tree Splitting](https://mlsanity.com/paper/2103.07845/) (2021)
- [Memorization and Generalization in Neural Code Intelligence Models](https://mlsanity.com/paper/2106.08704/) (2021)
- [An Extractive-and-Abstractive Framework for Source Code Summarization](https://mlsanity.com/paper/2206.07245/) (2022)
- [Assemble Foundation Models for Automatic Code Summarization](https://mlsanity.com/paper/2201.05222/) (2022)
- [One Adapter for All Programming Languages? Adapter Tuning for Code Search and Summarization](https://mlsanity.com/paper/2303.15822/) (2023)
- [CODE-MVP: Learning to Represent Source Code from Multiple Views with Contrastive Pre-Training](https://mlsanity.com/paper/2205.02029/) (2022)
- [Long-Range Modeling of Source Code Files with eWASH: Extended Window Access by Syntax Hierarchy](https://mlsanity.com/paper/2109.08780/) (2021)
- [Source Code Summarization with Structural Relative Position Guided Transformer](https://mlsanity.com/paper/2202.06521/) (2022)

## Abstract (arXiv)

Code summarization provides a high level natural language description of the function performed by code, as it can benefit the software maintenance, code categorization and retrieval. To the best of our knowledge, most state-of-the-art approaches follow an encoder-decoder framework which encodes the code into a hidden space and then decode it into natural language space, suffering from two major drawbacks: a) Their encoders only consider the sequential content of code, ignoring the tree structure which is also critical for the task of code summarization, b) Their decoders are typically trained to predict the next word by maximizing the likelihood of next ground-truth word with previous ground-truth word given. However, it is expected to generate the entire sequence from scratch at test time. This discrepancy can cause an textitexposure bias issue, making the learnt decoder suboptimal. In this paper, we incorporate an abstract syntax tree structure as well as sequential content of code snippets into a deep reinforcement learning framework (i.e., actor-critic network). The actor network provides the confidence of predicting the next word according to current state. On the other hand, the critic network evaluates the reward value of all possible extensions of the current state and can provide global guidance for explorations. We employ an advantage reward composed of BLEU metric to train both networks. Comprehensive experiments on a real-world dataset show the effectiveness of our proposed model when compared with some state-of-the-art methods.