Learning to Extend Program Graphs to Work-in-Progress Code
Not on the map. Why?
Abstract (arXiv)
Source code spends most of its time in a broken or incomplete state during software development. This presents a challenge to machine learning for code, since high-performing models typically rely on graph structured representations of programs derived from traditional program analyses. Such analyses may be undefined for broken or incomplete code. We extend the notion of program graphs to work-in-progress code by learning to predict edge relations between tokens, training on well-formed code before transferring to work-in-progress code. We consider the tasks of code completion and localizing and repairing variable misuse in a work-in-process scenario. We demonstrate that training relation-aware models with fine-tuned edges consistently leads to improved performance on both tasks.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 50 of its 85 references that are in the map. First posted on arXiv 28 May 2021.
Builds on
50 of 85 references.
References · 50
- 2017Attention Is All You Needcited by 59,932
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2014Adam: A Method for Stochastic Optimizationcited by 36,960
- 2016Layer Normalizationcited by 5,579
- 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
- 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
- 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
- 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
- 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,031
- 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,830
- 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,366
- 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,286
38 more
- 2016The Concrete Distribution: A Continuous Relaxation of Discrete Random Variablescited by 1,135
- 2018Self-Attention with Relative Position Representationscited by 1,112
- 2015Effective Approaches to Attention-based Neural Machine Translationcited by 1,024
- 2020On Layer Normalization in the Transformer Architecturecited by 906
- 2020GraphCodeBERT: Pre-training Code Representations with Data Flowcited by 600
- 2019RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parserscited by 420
- 2018Neural Relational Inference for Interacting Systemscited by 339
- 2020Universal Dependencies v2: An Evergrowing Multilingual Treebank Collectioncited by 269
- 2018code2vec: Learning Distributed Representations of Codecited by 249
- 2019Learning Discrete Structures for Graph Neural Networkscited by 228
- 2020IntelliCode Compose: Code Generation Using Transformercited by 225
- 2017A Survey of Machine Learning for Big Code and Naturalnesscited by 220
- 2017Learning to Represent Programs with Graphscited by 219
- 2018code2seq: Generating Sequences from Structured Representations of Codecited by 177
- 2018Tree-to-tree Neural Networks for Program Translationcited by 93
- 2020Graph-based, Self-Supervised Program Repair from Diagnostic Feedbackcited by 93
- 2017Abstract Syntax Networks for Code Generation and Semantic Parsingcited by 92
- 2016Neuro-Symbolic Program Synthesiscited by 84
- 2020Code Prediction by Feeding Trees to Transformerscited by 70
- 2020Gradient Estimation with Stochastic Softmax Trickscited by 66
- 2021Language-Agnostic Representation Learning of Source Code from Structure and Contextcited by 63
- 2018Generative Code Modeling with Graphscited by 59
- 2018Neural Code Comprehension: A Learnable Representation of Code Semanticscited by 59
- 2020Big Code != Big Vocabulary: Open-Vocabulary Models for Source Codecited by 53
- 2020Pointer Graph Networkscited by 52
- 2019Getafix: Learning to Fix Bugs Automaticallycited by 40
- 2020Fast and Memory-Efficient Neural Code Completioncited by 38
- 2015On the "Naturalness" of Buggy Codecited by 35
- 2020Learning to Execute Programs with Instruction Pointer Attention Graph Neural Networkscited by 35
- 2021CodeTrans: Towards Cracking the Language of Silicon's Code Through Self-Supervised Deep Learning and High Performance Computingcited by 28
- 2019Learning to Fix Build Errors with Graph2Diff Neural Networkscited by 26
- 2020ProGraML: Graph-based Deep Learning for Program Optimization and Analysiscited by 23
- 2018StructVAE: Tree-structured Latent Variable Models for Semi-supervised Semantic Parsingcited by 22
- 2014Structured Generative Models of Natural Source Codecited by 19
- 2018Open Vocabulary Learning on Source Code with a Graph-Structured Cachecited by 10
- 2020Learning Graph Structure With A Finite-State Automaton Layercited by 10
- 2020Towards Full-line Code Completion with Neural Language Modelscited by 6
- 2019Maybe Deep Neural Networks are the Best Choice for Modeling Source Codecited by 5
Built on by
Citing papers per year: 2022: 1; 2023: 1; 2024: 1.
3 papers.