mlsanityData to 7 Oct 2026

A Transformer-based Approach for Source Code Summarization

arXiv 2005.00653 ↗First posted 1 May 2020cs.SE cs.AI cs.LGcited by 129

Not on the map. Why?

Abstract (arXiv)

Generating a readable summary that describes the functionality of a program is known as source code summarization. In this task, learning code representation by modeling the pairwise relationship between code tokens to capture their long-range dependencies is crucial. To learn code representation for summarization, we explore the Transformer model that uses a self-attention mechanism and has shown to be effective in capturing long-range dependencies. In this work, we show that despite the approach is simple, it outperforms the state-of-the-art techniques by a significant margin. We perform extensive analysis and ablation studies that reveal several important findings, e.g., the absolute encoding of source code tokens' position hinders, while relative encoding significantly improves the summarization performance. We have made our code publicly available to facilitate future research.

Key facts. Cited by 129 papers in the map (data through 7 Oct 2026). First posted on arXiv 1 May 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 30; 2022: 27; 2023: 29; 2024: 18; 2025: 15; 2026: 9.

129 papers.

Sort:

By area:

  1. 2021Unified Pre-training for Program Understanding and Generationcited by 361
  2. 2023OctoPack: Instruction Tuning Code Large Language Modelscited by 178
  3. 2021Retrieval Augmented Code Generation and Summarizationcited by 148
  4. 2022ReACC: A Retrieval-Augmented Code Completion Frameworkcited by 117
  5. 2020Retrieval-Augmented Generation for Code Summarization via Hybrid GNNcited by 89
  6. 2023Automatic Semantic Augmentation of Language Model Prompts (for Code Summarization)cited by 72
  7. 2023Is ChatGPT the Ultimate Programming Assistant -- How far is it?cited by 72
  8. 2024Code-Aware Prompting: A study of Coverage Guided Test Generation in Regression Setting using LLMcited by 63
  9. 2021AVATAR: A Parallel Corpus for Java-Python Program Translationcited by 59
  10. 2022NatGen: Generative pre-training by "Naturalizing" source codecited by 53
  11. 2021On the Evaluation of Neural Code Summarizationcited by 47
  12. 2021Multilingual training for Software Engineeringcited by 43
38 more
  1. 2022MCoNaLa: A Benchmark for Code Generation from Multiple Natural Languagescited by 41
  2. 2021CoProtector: Protect Open-Source Code against Unauthorized Training Usage with Data Poisoningcited by 39
  3. 2025LongCodeZip: Compress Long Context for Code Language Modelscited by 38
  4. 2021What do pre-trained code models know about code?cited by 36
  5. 2024Can Large Language Models Write Parallel Code?cited by 36
  6. 2024Automatic Programming: Large Language Models and Beyondcited by 35
  7. 2023Automatic Code Summarization via ChatGPT: How Far Are We?cited by 34
  8. 2023Natural Language Generation and Understanding of Big Code for AI-Assisted Programming: A Reviewcited by 32
  9. 2023Learning Deep Semantics for Test Completioncited by 31
  10. 2021CodeQA: A Question Answering Dataset for Source Code Comprehensioncited by 30
  11. 2023Distilled GPT for Source Code Summarizationcited by 30
  12. 2024Tag-LLM: Repurposing General-Purpose LLMs for Specialized Domainscited by 30
  13. 2024Source Code Summarization in the Era of Large Language Modelscited by 30
  14. 2021Code Structure Guided Transformer for Source Code Summarizationcited by 29
  15. 2021On Multi-Modal Learning of Editing Source Codecited by 25
  16. 2021Symbolic Behaviour in Artificial Intelligencecited by 24
  17. 2021Project-Level Encoding for Neural Source Code Summarization of Subroutinescited by 24
  18. 2021Counterfactual Explanations for Models of Codecited by 24
  19. 2021CoDesc: A Large Code-Description Parallel Datasetcited by 23
  20. 2021A Survey on Machine Learning Techniques for Source Code Analysiscited by 23
  21. 2023Deep Learning for Code Intelligence: Survey, Benchmark and Toolkitcited by 22
  22. 2021VELVET: a noVel Ensemble Learning approach to automatically locate VulnErable sTatementscited by 20
  23. 2024Performance-Aligned LLMs for Generating Fast Codecited by 20
  24. 2021Improving Code Summarization with Block-wise Abstract Syntax Tree Splittingcited by 19
  25. 2024UniCoder: Scaling Code Large Language Model via Universal Codecited by 18
  26. 2022Assemble Foundation Models for Automatic Code Summarizationcited by 17
  27. 2023GrACE: Generation using Associated Code Editscited by 16
  28. 2022An Extractive-and-Abstractive Framework for Source Code Summarizationcited by 15
  29. 2022Source Code Summarization with Structural Relative Position Guided Transformercited by 14
  30. 2022M2TS: Multi-Scale Multi-Modal Approach Based on Transformer for Source Code Summarizationcited by 14
  31. 2022An Exploratory Study on Code Attention in BERTcited by 12
  32. 2021Impact of Evaluation Methodologies on Code Summarizationcited by 11
  33. 2021AST-Transformer: Encoding Abstract Syntax Trees Efficiently for Code Summarizationcited by 11
  34. 2023HPC-Coder: Modeling Parallel Programs using Large Language Modelscited by 11
  35. 2024Studying LLM Performance on Closed- and Open-source Datacited by 11
  36. 2021API2Com: On the Improvement of Automatically Generated Code Comments Using API Documentationscited by 10
  37. 2025Towards an Understanding of Context Utilization in Code Intelligencecited by 10
  38. 2021Leveraging Unsupervised Learning to Summarize APIs Discussed in Stack Overflowcited by 9

50 most cited of 129 All 129 (JSON)