# Code Completion with Neural Attention and Pointer Networks

arXiv: https://arxiv.org/abs/1711.09573 · HTML: https://mlsanity.com/paper/1711.09573/

## Key facts

- Cited by 49 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Nov 2017.

## Built on by (most cited first, 30 of 49)

- [Unified Pre-training for Program Understanding and Generation](https://mlsanity.com/paper/2103.06333/) (2021)
- [An Empirical Evaluation of Using Large Language Models for Automated Unit Test Generation](https://mlsanity.com/paper/2302.06527/) (2023)
- [Debug like a Human: A Large Language Model Debugger via Verifying Runtime Execution Step-by-step](https://mlsanity.com/paper/2402.16906/) (2024)
- [ReACC: A Retrieval-Augmented Code Completion Framework](https://mlsanity.com/paper/2203.07722/) (2022)
- [Execution-based Code Generation using Deep Reinforcement Learning](https://mlsanity.com/paper/2301.13816/) (2023)
- [Leveraging Automated Unit Tests for Unsupervised Code Translation](https://mlsanity.com/paper/2110.06773/) (2021)
- [DOBF: A Deobfuscation Pre-Training Objective for Programming Languages](https://mlsanity.com/paper/2102.07492/) (2021)
- [Language-Agnostic Representation Learning of Source Code from Structure and Context](https://mlsanity.com/paper/2103.11318/) (2021)
- [CodeFill: Multi-token Code Completion by Jointly Learning from Structure and Naming Sequences](https://mlsanity.com/paper/2202.06689/) (2022)
- [Compilable Neural Code Generation with Compiler Feedback](https://mlsanity.com/paper/2203.05132/) (2022)
- [RAP-Gen: Retrieval-Augmented Patch Generation with CodeT5 for Automatic Program Repair](https://mlsanity.com/paper/2309.06057/) (2023)
- [SpreadsheetCoder: Formula Prediction from Semi-structured Context](https://mlsanity.com/paper/2106.15339/) (2021)
- [Dataflow-Guided Retrieval Augmentation for Repository-Level Code Completion](https://mlsanity.com/paper/2405.19782/) (2024)
- [An LLM-Assisted Easy-to-Trigger Backdoor Attack on Code Completion Models: Injecting Disguised Vulnerabilities against Strong Detection](https://mlsanity.com/paper/2406.06822/) (2024)
- [StructCoder: Structure-Aware Transformer for Code Generation](https://mlsanity.com/paper/2206.05239/) (2022)
- [AST-T5: Structure-Aware Pretraining for Code Generation and Understanding](https://mlsanity.com/paper/2401.03003/) (2024)
- [Learning Deep Semantics for Test Completion](https://mlsanity.com/paper/2302.10166/) (2023)
- [Code Completion by Modeling Flattened Abstract Syntax Trees as Graphs](https://mlsanity.com/paper/2103.09499/) (2021)
- [Large Language Models of Code Fail at Completing Code with Potential Bugs](https://mlsanity.com/paper/2306.03438/) (2023)
- [A Survey on Machine Learning Techniques for Source Code Analysis](https://mlsanity.com/paper/2110.09610/) (2021)
- [InstructABSA: Instruction Learning for Aspect Based Sentiment Analysis](https://mlsanity.com/paper/2302.08624/) (2023)
- [Deep Learning for Code Intelligence: Survey, Benchmark and Toolkit](https://mlsanity.com/paper/2401.00288/) (2023)
- [Prompt-based Code Completion via Multi-Retrieval Augmented Generation](https://mlsanity.com/paper/2405.07530/) (2024)
- [Better Context Makes Better Code Language Models: A Case Study on Function Call Argument Completion](https://mlsanity.com/paper/2306.00381/) (2023)
- [Don't Complete It! Preventing Unhelpful Code Completion for Productive and Sustainable Neural Code Completion Systems](https://mlsanity.com/paper/2209.05948/) (2022)
- [CodeRAG: Finding Relevant and Necessary Knowledge for Retrieval-Augmented Repository-Level Code Completion](https://mlsanity.com/paper/2509.16112/) (2025)
- [Context-NER : Contextual Phrase Generation at Scale](https://mlsanity.com/paper/2109.08079/) (2021)
- [Does Your Neural Code Completion Model Use My Code? A Membership Inference Approach](https://mlsanity.com/paper/2404.14296/) (2024)
- [Syntax-Aware On-the-Fly Code Completion](https://mlsanity.com/paper/2211.04673/) (2022)
- [FT2Ra: A Fine-Tuning-Inspired Approach to Retrieval-Augmented Code Completion](https://mlsanity.com/paper/2404.01554/) (2024)

## Abstract (arXiv)

Intelligent code completion has become an essential research task to accelerate modern software development. To facilitate effective code completion for dynamically-typed programming languages, we apply neural language models by learning from large codebases, and develop a tailored attention mechanism for code completion. However, standard neural language models even with attention mechanism cannot correctly predict the out-of-vocabulary (OoV) words that restrict the code completion performance. In this paper, inspired by the prevalence of locally repeated terms in program source code, and the recently proposed pointer copy mechanism, we propose a pointer mixture network for better predicting OoV words in code completion. Based on the context, the pointer mixture network learns to either generate a within-vocabulary word through an RNN component, or regenerate an OoV word from local context through a pointer component. Experiments on two benchmarked datasets demonstrate the effectiveness of our attention mechanism and pointer mixture network on the code completion task.