A Survey on Natural Language Processing for Programming
Not on the map. Why?
Abstract (arXiv)
Natural language processing for programming aims to use NLP techniques to assist programming. It is increasingly prevalent for its effectiveness in improving productivity. Distinct from natural language, a programming language is highly structured and functional. Constructing a structure-based representation and a functionality-oriented algorithm is at the heart of program understanding and generation. In this paper, we conduct a systematic review covering tasks, datasets, evaluation methods, techniques, and models from the perspective of the structure-based and functionality-oriented property, aiming to understand the role of the two properties in each component. Based on the analysis, we illustrate unexplored areas and suggest potential directions for future work.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 63 of its 115 references that are in the map. First posted on arXiv 12 Dec 2022.
Builds on
63 of 115 references.
References · 63
- 2017Attention Is All You Needcited by 59,932
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2014Microsoft COCO: Common Objects in Contextcited by 17,498
- 2021Evaluating Large Language Models Trained on Codecited by 8,053
- 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
- 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
- 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
- 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
- 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
- 2021Program Synthesis with Large Language Modelscited by 3,509
- 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
38 more
- 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,813
- 2022Competition-Level Code Generation with AlphaCodecited by 1,427
- 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,286
- 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,028
- 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 890
- 2021Measuring Coding Challenge Competence With APPScited by 879
- 2021CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generationcited by 732
- 2019CodeSearchNet Challenge: Evaluating the State of Semantic Code Searchcited by 638
- 2020GraphCodeBERT: Pre-training Code Representations with Data Flowcited by 600
- 2022InCoder: A Generative Model for Code Infilling and Synthesiscited by 492
- 2020CodeBLEU: a Method for Automatic Evaluation of Code Synthesiscited by 468
- 2022DS-1000: A Natural and Reliable Benchmark for Data Science Code Generationcited by 442
- 2022CodeT: Code Generation with Generated Testscited by 369
- 2021Unified Pre-training for Program Understanding and Generationcited by 361
- 2016How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generationcited by 356
- 2022UniXcoder: Unified Cross-Modal Pre-training for Code Representationcited by 355
- 2022CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learningcited by 342
- 2019Devign: Effective Vulnerability Identification by Learning Comprehensive Program Semantics via Graph Neural Networkscited by 266
- 2020Unsupervised Translation of Programming Languagescited by 262
- 2020IntelliCode Compose: Code Generation Using Transformercited by 225
- 2017Learning to Represent Programs with Graphscited by 219
- 2021CodeNet: A Large-Scale AI for Code Dataset for Learning a Diversity of Coding Taskscited by 199
- 2018code2seq: Generating Sequences from Structured Representations of Codecited by 177
- 2022Language Models of Code are Few-Shot Commonsense Learnerscited by 160
- 2020A Transformer-based Approach for Source Code Summarizationcited by 129
- 2018Automated Vulnerability Detection in Source Code Using Deep Representation Learningcited by 114
- 2018An Empirical Study on Learning Bug-Fixing Patches in the Wild via Neural Machine Translationcited by 106
- 2018Mapping Language to Code in Programmatic Contextcited by 104
- 2022Language Models are General-Purpose Interfacescited by 96
- 2020Retrieval-Augmented Generation for Code Summarization via Hybrid GNNcited by 89
- 2018Leveraging Grammar and Reinforcement Learning for Neural Program Synthesiscited by 83
- 2020PyMT5: multi-mode translation of natural language and Python code with transformerscited by 80
- 2022Code4Struct: Code Generation for Few-Shot Event Structure Predictioncited by 70
- 2021Language-Agnostic Representation Learning of Source Code from Structure and Contextcited by 63
- 2019Code Generation as a Dual Task of Code Summarizationcited by 62
- 2016A Simple, Fast Diverse Decoding Algorithm for Neural Generationcited by 58
- 2017Code Completion with Neural Attention and Pointer Networkscited by 49
- 2022PanGu-Coder: Program Synthesis with Function-Level Language Modelingcited by 47
50 most cited of 63 All 63 (JSON)
Built on by
Citing papers per year: 2024: 1; 2025: 2.
3 papers.