# Code Completion using Neural Attention and Byte Pair Encoding

arXiv: https://arxiv.org/abs/2004.06343 · HTML: https://mlsanity.com/paper/2004.06343/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Apr 2020.

## Built on by (most cited first, 2 of 2)

- [Energy-Based Models for Code Generation under Compilability Constraints](https://mlsanity.com/paper/2106.04985/) (2021)
- [A Survey of Deep Learning Models for Structural Code Understanding](https://mlsanity.com/paper/2205.01293/) (2022)

## Abstract (arXiv)

In this paper, we aim to do code completion based on implementing a Neural Network from Li et. al.. Our contribution is that we use an encoding that is in-between character and word encoding called Byte Pair Encoding (BPE). We use this on the source code files treating them as natural text without first going through the abstract syntax tree (AST). We have implemented two models: an attention-enhanced LSTM and a pointer network, where the pointer network was originally introduced to solve out of vocabulary problems. We are interested to see if BPE can replace the need for the pointer network for code completion.