mlsanityData to 7 Oct 2026

Few-shot training LLMs for project-specific code-summarization

arXiv 2207.04237 ↗First posted 9 Jul 2022cs.SE cs.LGcited by 128

Not on the map. Why?

Abstract (arXiv)

Very large language models (LLMs), such as GPT-3 and Codex have achieved state-of-the-art performance on several natural-language tasks, and show great promise also for code. A particularly exciting aspect of LLMs is their knack for few-shot and zero-shot learning: they can learn to perform a task with very few examples. Few-shotting has particular synergies in software engineering, where there are a lot of phenomena (identifier names, APIs, terminology, coding patterns) that are known to be highly project-specific. However, project-specific data can be quite limited, especially early in the history of a project; thus the few-shot learning capacity of LLMs might be very relevant. In this paper, we investigate the use few-shot training with the very large GPT (Generative Pre-trained Transformer) Codex model, and find evidence suggesting that one can significantly surpass state-of-the-art models for code-summarization, leveraging project-specific training.

Key facts. Cited by 128 papers in the map (data through 7 Oct 2026). Builds on 17 of its 25 references that are in the map. First posted on arXiv 9 Jul 2022.

Builds on

17 of 25 references.

References · 17

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  6. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  7. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  8. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  9. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,286
  10. 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 890
  11. 2021CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generationcited by 732
  12. 2019CodeSearchNet Challenge: Evaluating the State of Semantic Code Searchcited by 638
5 more
  1. 2020GraphCodeBERT: Pre-training Code Representations with Data Flowcited by 600
  2. 2022InCoder: A Generative Model for Code Infilling and Synthesiscited by 492
  3. 2021Unified Pre-training for Program Understanding and Generationcited by 361
  4. 2021On the Evaluation of Neural Code Summarizationcited by 47
  5. 2021Multilingual training for Software Engineeringcited by 43

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 29; 2024: 53; 2025: 28; 2026: 16.

128 papers.

Sort:

By area:

  1. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,332
  2. 2023CacheGen: KV Cache Compression and Streaming for Fast Large Language Model Servingcited by 199
  3. 2023Automatic Semantic Augmentation of Language Model Prompts (for Code Summarization)cited by 72
  4. 2023Is ChatGPT the Ultimate Programming Assistant -- How far is it?cited by 72
  5. 2024Vul-RAG: Enhancing LLM-based Vulnerability Detection via Knowledge-level RAGcited by 67
  6. 2024EffiBench: Benchmarking the Efficiency of Automatically Generated Codecited by 62
  7. 2024Exploring and Unleashing the Power of Large Language Models in Automated Code Translationcited by 57
  8. 2023Bias Testing and Mitigation in LLM-based Code Generationcited by 53
  9. 2024A Systematic Survey of Text Summarization: From Statistical Methods to Large Language Modelscited by 52
  10. 2023ICE-Score: Instructing Large Language Models to Evaluate Codecited by 50
  11. 2023Large Language Models and Simple, Stupid Bugscited by 45
  12. 2024Can LLMs Replace Manual Annotation of Software Engineering Artifacts?cited by 40
38 more
  1. 2023CodeScope: An Execution-based Multilingual Multitask Multidimensional Benchmark for Evaluating LLMs on Code Understanding and Generationcited by 39
  2. 2023Recommending Root-Cause and Mitigation Steps for Cloud Incidents using Large Language Modelscited by 38
  3. 2024Can Large Language Models Understand Symbolic Graphics Programs?cited by 32
  4. 2024INMS: Memory Sharing for Large Language Model based Agentscited by 31
  5. 2024Source Code Summarization in the Era of Large Language Modelscited by 30
  6. 2023Better patching using LLM prompting, via Self-Consistencycited by 29
  7. 2023Exploring Parameter-Efficient Fine-Tuning Techniques for Code Generation with Large Language Modelscited by 28
  8. 2024EffiLearner: Enhancing Efficiency of Generated Code via Self-Optimizationcited by 28
  9. 2024RMCBench: Benchmarking Large Language Models' Resistance to Malicious Codecited by 27
  10. 2024MHPP: Exploring the Capabilities and Limitations of Language Models Beyond Basic Code Generationcited by 25
  11. 2023A Comparative Analysis of Large Language Models for Code Documentation Generationcited by 24
  12. 2025EffiBench-X: A Multi-Language Benchmark for Measuring Efficiency of LLM-Generated Codecited by 24
  13. 2024Applicability of Large Language Models and Generative Models for Legal Case Judgement Summarizationcited by 23
  14. 2024MECD: Unlocking Multi-Event Causal Discovery in Video Reasoningcited by 23
  15. 2023Trustworthy and Synergistic Artificial Intelligence for Software Engineering: Vision and Roadmapscited by 19
  16. 2024X-lifecycle Learning for Cloud Incident Management using LLMscited by 19
  17. 2024NaturalCodeBench: Examining Coding Performance Mismatch on HumanEval and Natural User Promptscited by 19
  18. 2023From Commit Message Generation to History-Aware Commit Message Completioncited by 16
  19. 2024Special Characters Attack: Toward Scalable Training Data Extraction From Large Language Modelscited by 16
  20. 2023CADTalk: An Algorithm and Benchmark for Semantic Commenting of CAD Programscited by 15
  21. 2024DrugLLM: Open Large Language Model for Few-shot Molecule Generationcited by 15
  22. 2024EffiCoder: Enhancing Code Generation in Large Language Models through Efficiency-Aware Fine-tuningcited by 15
  23. 2022Psychologically-informed chain-of-thought prompts for metaphor understanding in large language modelscited by 14
  24. 2025Beyond Final Code: A Process-Oriented Error Analysis of Software Development Agents in Real-World GitHub Scenarioscited by 14
  25. 2023SecureFalcon: Are We There Yet in Automated Software Vulnerability Detection with LLMs?cited by 13
  26. 2023Unit Test Generation using Generative AI : A Comparative Performance Analysis of Autogeneration Toolscited by 13
  27. 2024Calibration of Large Language Models on Code Summarizationcited by 12
  28. 2023The potential of LLMs for coding with low-resource and domain-specific programming languagescited by 11
  29. 2024Studying LLM Performance on Closed- and Open-source Datacited by 11
  30. 2025Enhancing LLM Performance Through Debate: An Empirical Study on Multi-Agent Debate for Coding Taskscited by 11
  31. 2024Integrating Chemistry Knowledge in Large Language Models via Prompt Engineeringcited by 10
  32. 2024RepoMasterEval: Evaluating Code Completion via Real-World Repositoriescited by 10
  33. 2025A Contemporary Survey of Large Language Model Assisted Program Analysiscited by 10
  34. 2023Multilingual Code Co-Evolution Using Large Language Modelscited by 9
  35. 2023A Prompt Learning Framework for Source Code Summarizationcited by 9
  36. 2024Natural Language Outlines for Code: Literate Programming in the LLM Eracited by 9
  37. 2024Measuring the Influence of Incorrect Code on Test Generationcited by 9
  38. 2025Zero-shot Meta-learning for Tabular Prediction Tasks with Adversarially Pre-trained Transformercited by 8

50 most cited of 128 All 128 (JSON)