Prompt-Driven Code Summarization: A Systematic Literature Review
Not on the map. Why?
Abstract (arXiv)
Software documentation is essential for program comprehension, developer onboarding, code review, and long-term maintenance. Yet producing quality documentation manually is time-consuming and frequently yields incomplete or inconsistent results. Large language models (LLMs) offer a promising solution by automatically generating natural language descriptions from source code, helping developers understand code more efficiently, facilitating maintenance, and supporting downstream activities such as defect localization and commit message generation. However, the effectiveness of LLMs in documentation tasks critically depends on how they are prompted. Properly structured instructions can substantially improve model performance, making prompt engineering-the design of input prompts to guide model behavior-a foundational technique in LLM-based software engineering. Approaches such as few-shot prompting, chain-of-thought reasoning, retrieval-augmented generation, and zero-shot learning show promise for code summarization, yet current research remains fragmented. There is limited understanding of which prompting strategies work best, for which models, and under what conditions. Moreover, evaluation practices vary widely, with most studies relying on overlap-based metrics that may not capture semantic quality. This systematic literature review consolidates existing evidence, categorizes prompting paradigms, examines their effectiveness, and identifies gaps to guide future research and practical adoption.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 66 of its 114 references that are in the map. First posted on arXiv 16 Apr 2026.
Builds on
66 of 114 references.
References · 66
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2023GPT-4 Technical Reportcited by 18,396
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
- 2021Evaluating Large Language Models Trained on Codecited by 8,320
- 2017Deep reinforcement learning from human preferencescited by 3,912
- 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
- 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
- 2023Code Llama: Open Foundation Models for Codecited by 2,139
- 2022Competition-Level Code Generation with AlphaCodecited by 1,462
- 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,310
38 more
- 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,181
- 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,045
- 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 906
- 2022Automatic Chain of Thought Prompting in Large Language Modelscited by 688
- 2020GraphCodeBERT: Pre-training Code Representations with Data Flowcited by 609
- 2022InCoder: A Generative Model for Code Infilling and Synthesiscited by 498
- 2023Large Language Models for Software Engineering: A Systematic Literature Reviewcited by 399
- 2021Unified Pre-training for Program Understanding and Generationcited by 366
- 2022A Systematic Evaluation of Large Language Models of Codecited by 363
- 2023CodeT5+: Open Code Large Language Models for Code Understanding and Generationcited by 333
- 2021Do Prompt-Based Models Really Understand the Meaning of their Prompts?cited by 326
- 2021Retrieval Augmented Code Generation and Summarizationcited by 149
- 2022Few-shot training LLMs for project-specific code-summarizationcited by 129
- 2023An Analysis of the Automatic Bug Fixing Performance of ChatGPTcited by 110
- 2020CoDEx: A Comprehensive Knowledge Graph Completion Benchmarkcited by 102
- 2016A Convolutional Attention Network for Extreme Summarization of Source Codecited by 91
- 2020Retrieval-Augmented Generation for Code Summarization via Hybrid GNNcited by 89
- 2020Improved Code Summarization via a Graph Neural Networkcited by 79
- 2023Large Language Models are Few-Shot Summarizers: Multi-Intent Comment Generation via In-Context Learningcited by 72
- 2023Unleashing the potential of prompt engineering for large language modelscited by 72
- 2024From LLM to NMT: Advancing Low-Resource Machine Translation with Claudecited by 58
- 2021On the Evaluation of Neural Code Summarizationcited by 48
- 2023What Makes Good In-context Demonstrations for Code Intelligence Tasks with LLMs?cited by 47
- 2024A Survey of Prompt Engineering Methods in Large Language Models for Different NLP Taskscited by 47
- 2017Guidelines for including grey literature and conducting multivocal literature reviews in software engineeringcited by 41
- 2022Automatic Code Documentation Generation Using GPT-3cited by 37
- 2019Retrieve and Refine: Exemplar-based Neural Comment Generationcited by 36
- 2020A Systematic Literature Review on the Use of Deep Learning in Software Engineering Researchcited by 35
- 2023How are Prompts Different in Terms of Sensitivity?cited by 35
- 2023Automatic Code Summarization via ChatGPT: How Far Are We?cited by 34
- 2024Source Code Summarization in the Era of Large Language Modelscited by 31
- 2022ERNIE-Code: Beyond English-Centric Cross-lingual Pretraining for Programming Languagescited by 29
- 2021Generating Adversarial Computer Programs using Optimized Obfuscationscited by 22
- 2023Evaluating Code Summarization Techniques: A New Metric and an Empirical Characterizationcited by 21
- 2024Do Advanced Language Models Eliminate the Need for Prompt Engineering in Software Engineering?cited by 19
- 2019Automatic Code Summarization: A Systematic Literature Reviewcited by 18
- 2025On the Effectiveness of LLM-as-a-judge for Code Generation and Summarizationcited by 18
- 2024Efficient and Green Large Language Models for Software Engineering: Literature Review, Vision, and the Road Aheadcited by 17
50 most cited of 66 All 66 (JSON)
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024The Impact of Prompt Programming on Function-Level Code Generation
- 2024Can Large Language Models Serve as Evaluators for Code Summarization?
- 2025Do Prompt Patterns Affect Code Quality? A First Empirical Assessment of ChatGPT-Generated Code
- 2024Understanding Defects in Generated Codes by Language Models
- 2026Failure-Aware Enhancements for Large Language Model (LLM) Code Generation: An Empirical Study on Decision Framework ↗
- 2025Prompt Engineering for Requirements Engineering: A Literature Review and Roadmap
- 2025Are Prompts All You Need? Evaluating Prompt-Based Large Language Models (LLM)s for Software Requirements Classification
- 2025Prompt engineering and framework: implementation to increase code reliability based guideline for LLMs