# Code Summarization with Structure-induced Transformer

arXiv: https://arxiv.org/abs/2012.14710 · HTML: https://mlsanity.com/paper/2012.14710/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Dec 2020.

## Built on by (most cited first, 29 of 29)

- [On the Evaluation of Neural Code Summarization](https://mlsanity.com/paper/2107.07112/) (2021)
- [Automatic Programming: Large Language Models and Beyond](https://mlsanity.com/paper/2405.02213/) (2024)
- [Automatic Code Summarization via ChatGPT: How Far Are We?](https://mlsanity.com/paper/2305.12865/) (2023)
- [Source Code Summarization in the Era of Large Language Models](https://mlsanity.com/paper/2407.07959/) (2024)
- [Code Structure Guided Transformer for Source Code Summarization](https://mlsanity.com/paper/2104.09340/) (2021)
- [Deep Learning for Code Intelligence: Survey, Benchmark and Toolkit](https://mlsanity.com/paper/2401.00288/) (2023)
- [Analyzing the Performance of Large Language Models on Code Summarization](https://mlsanity.com/paper/2404.08018/) (2024)
- [An Extractive-and-Abstractive Framework for Source Code Summarization](https://mlsanity.com/paper/2206.07245/) (2022)
- [Assemble Foundation Models for Automatic Code Summarization](https://mlsanity.com/paper/2201.05222/) (2022)
- [Source Code Summarization with Structural Relative Position Guided Transformer](https://mlsanity.com/paper/2202.06521/) (2022)
- [CP-BCS: Binary Code Summarization Guided by Control Flow Graph and Pseudo Code](https://mlsanity.com/paper/2310.16853/) (2023)
- [HierarchyNet: Learning to Summarize Source Code with Heterogeneous Representations](https://mlsanity.com/paper/2205.15479/) (2022)
- [A Prompt Learning Framework for Source Code Summarization](https://mlsanity.com/paper/2312.16066/) (2023)
- [EyeTrans: Merging Human and Machine Attention for Neural Code Summarization](https://mlsanity.com/paper/2402.14096/) (2024)
- [Adversarial Self-Attention for Language Understanding](https://mlsanity.com/paper/2206.12608/) (2022)
- [ESALE: Enhancing Code-Summary Alignment Learning for Source Code Summarization](https://mlsanity.com/paper/2407.01646/) (2024)
- [Tram: A Token-level Retrieval-augmented Mechanism for Source Code Summarization](https://mlsanity.com/paper/2305.11074/) (2023)
- [HDT: Hierarchical Document Transformer](https://mlsanity.com/paper/2407.08330/) (2024)
- [Large Language Models are Qualified Benchmark Builders: Rebuilding Pre-Training Datasets for Advancing Code Intelligence Tasks](https://mlsanity.com/paper/2504.19444/) (2025)
- [Not All Attention Is All You Need](https://mlsanity.com/paper/2104.04692/) (2021)
- [GypSum: Learning Hybrid Representations for Code Summarization](https://mlsanity.com/paper/2204.12916/) (2022)
- [Abstract Syntax Tree for Programming Language Understanding and Representation: How Far Are We?](https://mlsanity.com/paper/2312.00413/) (2023)
- [Instruction-Driven Game Engines on Large Language Models](https://mlsanity.com/paper/2404.00276/) (2024)
- [Semantic-Preserving Adversarial Code Comprehension](https://mlsanity.com/paper/2209.05130/) (2022)
- [Knowledge Transfer for Pseudo-code Generation from Low Resource Programming Language](https://mlsanity.com/paper/2303.09062/) (2023)
- [Variational Prefix Tuning for Diverse and Accurate Code Summarization Using Pre-trained Language Models](https://mlsanity.com/paper/2505.09062/) (2025)
- [Understanding Long Programming Languages with Structure-Aware Sparse Attention](https://mlsanity.com/paper/2205.13730/) (2022)
- [KEN: Kernel Extensions using Natural Language](https://mlsanity.com/paper/2312.05531/) (2023)
- [PLMGH: What Matters in PLM-GNN Hybrids for Code Classification and Vulnerability Detection](https://mlsanity.com/paper/2604.25599/) (2026)

## Abstract (arXiv)

Code summarization (CS) is becoming a promising area in recent language understanding, which aims to generate sensible human language automatically for programming language in the format of source code, serving in the most convenience of programmer developing. It is well known that programming languages are highly structured. Thus previous works attempt to apply structure-based traversal (SBT) or non-sequential models like Tree-LSTM and graph neural network (GNN) to learn structural program semantics. However, it is surprising that incorporating SBT into advanced encoder like Transformer instead of LSTM has been shown no performance gain, which lets GNN become the only rest means modeling such necessary structural clue in source code. To release such inconvenience, we propose structure-induced Transformer, which encodes sequential code inputs with multi-view structural clues in terms of a newly-proposed structure-induced self-attention mechanism. Extensive experiments show that our proposed structure-induced Transformer helps achieve new state-of-the-art results on benchmarks.