# Code Completion by Modeling Flattened Abstract Syntax Trees as Graphs

arXiv: https://arxiv.org/abs/2103.09499 · HTML: https://mlsanity.com/paper/2103.09499/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 39 references that are in the map.
- First posted on arXiv 17 Mar 2021.

## Builds on (most cited first, 28 of 28)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [A Comprehensive Survey on Graph Neural Networks](https://mlsanity.com/paper/1901.00596/) (2019)
- [Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context](https://mlsanity.com/paper/1901.02860/) (2019)
- [Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics](https://mlsanity.com/paper/1705.07115/) (2017)
- [Pointer Networks](https://mlsanity.com/paper/1506.03134/) (2015)
- [GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks](https://mlsanity.com/paper/1711.02257/) (2017)
- [code2vec: Learning Distributed Representations of Code](https://mlsanity.com/paper/1803.09473/) (2018)
- [IntelliCode Compose: Code Generation Using Transformer](https://mlsanity.com/paper/2005.08025/) (2020)
- [A Survey of Machine Learning for Big Code and Naturalness](https://mlsanity.com/paper/1709.06182/) (2017)
- [Learning to Represent Programs with Graphs](https://mlsanity.com/paper/1711.00740/) (2017)
- [code2seq: Generating Sequences from Structured Representations of Code](https://mlsanity.com/paper/1808.01400/) (2018)
- [Improved Code Summarization via a Graph Neural Network](https://mlsanity.com/paper/2004.02843/) (2020)
- [Code Prediction by Feeding Trees to Transformers](https://mlsanity.com/paper/2003.13848/) (2020)
- [Generative Code Modeling with Graphs](https://mlsanity.com/paper/1805.08490/) (2018)
- [Pythia: AI-assisted Code Completion System](https://mlsanity.com/paper/1912.00742/) (2019)
- [Deep Learning for Source Code Modeling and Generation: Models, Applications and Challenges](https://mlsanity.com/paper/2002.05442/) (2020)
- [Code Completion with Neural Attention and Pointer Networks](https://mlsanity.com/paper/1711.09573/) (2017)
- [Fast and Memory-Efficient Neural Code Completion](https://mlsanity.com/paper/2004.13651/) (2020)
- [A General Path-Based Representation for Predicting Program Properties](https://mlsanity.com/paper/1803.09544/) (2018)
- [Multi-Task Learning for Dense Prediction Tasks: A Survey](https://mlsanity.com/paper/2004.13379/) (2020)
- [A Self-Attentional Neural Architecture for Code Completion with Multi-Task Learning](https://mlsanity.com/paper/1909.06983/) (2019)
- [Learning Python Code Suggestion with a Sparse Pointer Network](https://mlsanity.com/paper/1611.08307/) (2016)
- [Sequence Model Design for Code Completion in the Modern IDE](https://mlsanity.com/paper/2004.05249/) (2020)
- [TreeCaps: Tree-Structured Capsule Networks for Program Source Code Processing](https://mlsanity.com/paper/1910.12306/) (2019)
- [Improving the Robustness to Data Inconsistency between Training and Testing for Code Completion by Hierarchical Language Model](https://mlsanity.com/paper/2003.08080/) (2020)

## Built on by (most cited first, 30 of 30)

- [A Survey on Large Language Models for Code Generation](https://mlsanity.com/paper/2406.00515/) (2024)
- [Execution-based Code Generation using Deep Reinforcement Learning](https://mlsanity.com/paper/2301.13816/) (2023)
- [LLM Hallucinations in Practical Code Generation: Phenomena, Mechanism, and Mitigation](https://mlsanity.com/paper/2409.20550/) (2024)
- [Agents in Software Engineering: Survey, Landscape, and Vision](https://mlsanity.com/paper/2409.09030/) (2024)
- [GraphCoder: Enhancing Repository-Level Code Completion via Code Context Graph-based Retrieval and Language Model](https://mlsanity.com/paper/2406.07003/) (2024)
- [Natural Language Generation and Understanding of Big Code for AI-Assisted Programming: A Review](https://mlsanity.com/paper/2307.02503/) (2023)
- [CoCoSoDa: Effective Contrastive Learning for Code Search](https://mlsanity.com/paper/2204.03293/) (2022)
- [Deep Learning for Code Intelligence: Survey, Benchmark and Toolkit](https://mlsanity.com/paper/2401.00288/) (2023)
- [What to Retrieve for Effective Retrieval-Augmented Code Generation? An Empirical Study and Beyond](https://mlsanity.com/paper/2503.20589/) (2025)
- [Beyond Functional Correctness: Investigating Coding Style Inconsistencies in Large Language Models](https://mlsanity.com/paper/2407.00456/) (2024)
- [CodeRAG: Finding Relevant and Necessary Knowledge for Retrieval-Augmented Repository-Level Code Completion](https://mlsanity.com/paper/2509.16112/) (2025)
- [Does Your Neural Code Completion Model Use My Code? A Membership Inference Approach](https://mlsanity.com/paper/2404.14296/) (2024)
- [Syntax-Aware On-the-Fly Code Completion](https://mlsanity.com/paper/2211.04673/) (2022)
- [Investigating the Performance of Language Models for Completing Code in Functional Programming Languages: a Haskell Case Study](https://mlsanity.com/paper/2403.15185/) (2024)
- [AlignCoder: Aligning Retrieval with Target Intent for Repository-Level Code Completion](https://mlsanity.com/paper/2601.19697/) (2026)
- [CAT-probing: A Metric-based Approach to Interpret How Pre-trained Models for Programming Language Attend Code Structure](https://mlsanity.com/paper/2210.04633/) (2022)
- [A Neural Network Architecture for Program Understanding Inspired by Human Behaviors](https://mlsanity.com/paper/2206.04730/) (2022)
- [A Survey of Deep Learning Models for Structural Code Understanding](https://mlsanity.com/paper/2205.01293/) (2022)
- [A Library for Representing Python Programs as Graphs for Machine Learning](https://mlsanity.com/paper/2208.07461/) (2022)
- [Code Search Debiasing:Improve Search Results beyond Overall Ranking Performance](https://mlsanity.com/paper/2311.14901/) (2023)
- [Structured Code Representations Enable Data-Efficient Adaptation of Code Language Models](https://mlsanity.com/paper/2401.10716/) (2024)
- [AnyBipe: An End-to-End Framework for Training and Deploying Bipedal Robots Guided by Large Language Models](https://mlsanity.com/paper/2409.08904/) (2024)
- [Renaissance of Literate Programming in the Era of LLMs: Enhancing LLM-Based Code Generation in Large-Scale Projects](https://mlsanity.com/paper/2502.17441/) (2024)
- [DRAINCODE: Stealthy Energy Consumption Attacks on Retrieval-Augmented Code Generation via Context Poisoning](https://mlsanity.com/paper/2601.20615/) (2026)
- [CCCI: Code Completion with Contextual Information for Complex Data Transfer Tasks Using Large Language Models](https://mlsanity.com/paper/2503.23231/) (2025)
- [PhoenixRepair: Rethinking Repair Strategy Exploration in Software Agents](https://mlsanity.com/paper/2607.18859/) (2026)
- [FuncEvalGMN: Evaluating Functional Correctness of SQL via Graph Matching Network](https://mlsanity.com/paper/2407.14530/) (2024)
- [SwiftEval: Developing a Language-Specific Benchmark for LLM-generated Code Evaluation](https://mlsanity.com/paper/2505.24324/) (2025)
- [A Matter of Representation: Towards Graph-Based Abstract Code Generation](https://mlsanity.com/paper/2510.13163/) (2025)
- [Adaptive Critical Token-Aware Retrieval for Repository-Level Code Generation](https://mlsanity.com/paper/2609.01601/) (2026)

## Abstract (arXiv)

Code completion has become an essential component of integrated development environments. Contemporary code completion methods rely on the abstract syntax tree (AST) to generate syntactically correct code. However, they cannot fully capture the sequential and repetitive patterns of writing code and the structural information of the AST. To alleviate these problems, we propose a new code completion approach named CCAG, which models the flattened sequence of a partial AST as an AST graph. CCAG uses our proposed AST Graph Attention Block to capture different dependencies in the AST graph for representation learning in code completion. The sub-tasks of code completion are optimized via multi-task learning in CCAG, and the task balance is automatically achieved using uncertainty without the need to tune task weights. The experimental results show that CCAG has superior performance than state-of-the-art approaches and it is able to provide intelligent code completion.