# Retrieval-Augmented Generation for Code Summarization via Hybrid GNN

arXiv: https://arxiv.org/abs/2006.05405 · HTML: https://mlsanity.com/paper/2006.05405/

## Key facts

- Cited by 89 papers in the map (data through 14 Oct 2026).
- Placed in RAG for code, one of 15 directions in Retrieval-augmented generation.
- Builds on 34 of its 56 references that are in the map.
- First posted on arXiv 9 Jun 2020.

## Placed in

- Retrieval-augmented generation / [RAG for code](https://mlsanity.com/rag/rag-for-code/)

## Builds on (most cited first, 30 of 34)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Effective Approaches to Attention-based Neural Machine Translation](https://mlsanity.com/paper/1508.04025/) (2015)
- [Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks](https://mlsanity.com/paper/1506.03099/) (2015)
- [Gated Graph Sequence Neural Networks](https://mlsanity.com/paper/1511.05493/) (2015)
- [Measuring and Relieving the Over-smoothing Problem for Graph Neural Networks from the Topological View](https://mlsanity.com/paper/1909.03211/) (2019)
- [DeepGCNs: Can GCNs Go as Deep as CNNs?](https://mlsanity.com/paper/1904.03751/) (2019)
- [PairNorm: Tackling Oversmoothing in GNNs](https://mlsanity.com/paper/1909.12223/) (2019)
- [Devign: Effective Vulnerability Identification by Learning Comprehensive Program Semantics via Graph Neural Networks](https://mlsanity.com/paper/1909.03496/) (2019)
- [A Survey of Machine Learning for Big Code and Naturalness](https://mlsanity.com/paper/1709.06182/) (2017)
- [Learning to Represent Programs with Graphs](https://mlsanity.com/paper/1711.00740/) (2017)
- [code2seq: Generating Sequences from Structured Representations of Code](https://mlsanity.com/paper/1808.01400/) (2018)
- [DeepGauge: Multi-Granularity Testing Criteria for Deep Learning Systems](https://mlsanity.com/paper/1803.07519/) (2018)
- [A Transformer-based Approach for Source Code Summarization](https://mlsanity.com/paper/2005.00653/) (2020)
- [The Adverse Effects of Code Duplication in Machine Learning Models of Code](https://mlsanity.com/paper/1812.06469/) (2018)
- [Improving Automatic Source Code Summarization via Deep Reinforcement Learning](https://mlsanity.com/paper/1811.07234/) (2018)
- [Improved Code Summarization via a Graph Neural Network](https://mlsanity.com/paper/2004.02843/) (2020)
- [Code Generation as a Dual Task of Code Summarization](https://mlsanity.com/paper/1910.05923/) (2019)
- [Graph-to-Sequence Learning using Gated Graph Neural Networks](https://mlsanity.com/paper/1806.09835/) (2018)
- [A parallel corpus of Python functions and documentation strings for automated code documentation and code generation](https://mlsanity.com/paper/1707.02275/) (2017)
- [Reinforcement Learning Based Graph-to-Sequence Model for Natural Question Generation](https://mlsanity.com/paper/1908.04942/) (2019)
- [Structured Neural Summarization](https://mlsanity.com/paper/1811.01824/) (2018)
- [Learning Conditioned Graph Structures for Interpretable Visual Question Answering](https://mlsanity.com/paper/1806.07243/) (2018)
- [Automatically Generating Commit Messages from Diffs using Neural Machine Translation](https://mlsanity.com/paper/1708.09492/) (2017)
- [GraphFlow: Exploiting Conversation Flow with Graph Neural Networks for Conversational Machine Comprehension](https://mlsanity.com/paper/1908.00059/) (2019)
- [Graph2Seq: Graph to Sequence Learning with Attention-based Neural Networks](https://mlsanity.com/paper/1804.00823/) (2018)

## Built on by (most cited first, 30 of 89)

- [A Survey on Large Language Models for Code Generation](https://mlsanity.com/paper/2406.00515/) (2024)
- [Retrieval-Augmented Generation for AI-Generated Content: A Survey](https://mlsanity.com/paper/2402.19473/) (2024)
- [Retrieval Augmented Code Generation and Summarization](https://mlsanity.com/paper/2108.11601/) (2021)
- [Physics of Language Models: Part 3.2, Knowledge Manipulation](https://mlsanity.com/paper/2309.14402/) (2023)
- [Retrieving Multimodal Information for Augmented Generation: A Survey](https://mlsanity.com/paper/2303.10868/) (2023)
- [Enhancing Financial Sentiment Analysis via Retrieval Augmented Large Language Models](https://mlsanity.com/paper/2310.04027/) (2023)
- [LitLLM: A Toolkit for Scientific Literature Review](https://mlsanity.com/paper/2402.01788/) (2024)
- [NatGen: Generative pre-training by "Naturalizing" source code](https://mlsanity.com/paper/2206.07585/) (2022)
- [ASIF: Coupled Data Turns Unimodal Models to Multimodal Without Training](https://mlsanity.com/paper/2210.01738/) (2022)
- [xCodeEval: A Large Scale Multilingual Multitask Benchmark for Code Understanding, Generation, Translation and Retrieval](https://mlsanity.com/paper/2303.03004/) (2023)
- [No More Fine-Tuning? An Experimental Evaluation of Prompt Tuning in Code Intelligence](https://mlsanity.com/paper/2207.11680/) (2022)
- [Dataflow-Guided Retrieval Augmentation for Repository-Level Code Completion](https://mlsanity.com/paper/2405.19782/) (2024)
- [What is it like to program with artificial intelligence?](https://mlsanity.com/paper/2208.06213/) (2022)
- [OptiMUS-0.3: Using Large Language Models to Model and Solve Optimization Problems at Scale](https://mlsanity.com/paper/2407.19633/) (2024)
- [CodeXEmbed: A Generalist Embedding Model Family for Multiligual and Multi-task Code Retrieval](https://mlsanity.com/paper/2411.12644/) (2024)
- [Distilled GPT for Source Code Summarization](https://mlsanity.com/paper/2308.14731/) (2023)
- [Source Code Summarization in the Era of Large Language Models](https://mlsanity.com/paper/2407.07959/) (2024)
- [Code Structure Guided Transformer for Source Code Summarization](https://mlsanity.com/paper/2104.09340/) (2021)
- [On the Transferability of Pre-trained Language Models for Low-Resource Programming Languages](https://mlsanity.com/paper/2204.09653/) (2022)
- [Exploring Parameter-Efficient Fine-Tuning Techniques for Code Generation with Large Language Models](https://mlsanity.com/paper/2308.10462/) (2023)
- [RAG and RAU: A Survey on Retrieval-Augmented Language Model in Natural Language Processing](https://mlsanity.com/paper/2404.19543/) (2024)
- [Project-Level Encoding for Neural Source Code Summarization of Subroutines](https://mlsanity.com/paper/2103.11599/) (2021)
- [Data-centric Graph Learning: A Survey](https://mlsanity.com/paper/2310.04987/) (2023)
- [Deep Learning for Code Intelligence: Survey, Benchmark and Toolkit](https://mlsanity.com/paper/2401.00288/) (2023)
- [Assemble Foundation Models for Automatic Code Summarization](https://mlsanity.com/paper/2201.05222/) (2022)
- [GraphSearchNet: Enhancing GNNs via Capturing Global Dependencies for Semantic Code Search](https://mlsanity.com/paper/2111.02671/) (2021)
- [Object-Centric Multiple Object Tracking](https://mlsanity.com/paper/2309.00233/) (2023)
- [Source Code Summarization with Structural Relative Position Guided Transformer](https://mlsanity.com/paper/2202.06521/) (2022)
- [Accelerating Retrieval-Augmented Generation](https://mlsanity.com/paper/2412.15246/) (2024)
- [Reasoning in Trees: Improving Retrieval-Augmented Generation for Multi-Hop Question Answering](https://mlsanity.com/paper/2601.11255/) (2026)

## Abstract (arXiv)

Source code summarization aims to generate natural language summaries from structured code snippets for better understanding code functionalities. However, automatic code summarization is challenging due to the complexity of the source code and the language gap between the source code and natural language summaries. Most previous approaches either rely on retrieval-based (which can take advantage of similar examples seen from the retrieval database, but have low generalization performance) or generation-based methods (which have better generalization performance, but cannot take advantage of similar examples). This paper proposes a novel retrieval-augmented mechanism to combine the benefits of both worlds. Furthermore, to mitigate the limitation of Graph Neural Networks (GNNs) on capturing global graph structure information of source code, we propose a novel attention-based dynamic graph to complement the static graph representation of the source code, and design a hybrid message passing GNN for capturing both the local and global structural information. To evaluate the proposed approach, we release a new challenging benchmark, crawled from diversified large-scale open-source C projects (total 95k+ unique functions in the dataset). Our method achieves the state-of-the-art performance, improving existing methods by 1.42, 2.44 and 1.29 in terms of BLEU-4, ROUGE-L and METEOR.