# Abstract Syntax Tree for Programming Language Understanding and Representation: How Far Are We? arXiv: https://arxiv.org/abs/2312.00413 ยท HTML: https://mlsanity.com/paper/2312.00413/ ## Key facts - Cited by 5 papers in the map (data through 14 Oct 2026). - Builds on 45 of its 102 references that are in the map. - First posted on arXiv 1 Dec 2023. ## Builds on (most cited first, 30 of 45) - [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016) - [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014) - [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014) - [Microsoft COCO Captions: Data Collection and Evaluation Server](https://mlsanity.com/paper/1504.00325/) (2015) - [CodeBERT: A Pre-Trained Model for Programming and Natural Languages](https://mlsanity.com/paper/2002.08155/) (2020) - [CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation](https://mlsanity.com/paper/2102.04664/) (2021) - [CodeSearchNet Challenge: Evaluating the State of Semantic Code Search](https://mlsanity.com/paper/1909.09436/) (2019) - [UniXcoder: Unified Cross-Modal Pre-training for Code Representation](https://mlsanity.com/paper/2203.03850/) (2022) - [Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks](https://mlsanity.com/paper/1503.00075/) (2015) - [Devign: Effective Vulnerability Identification by Learning Comprehensive Program Semantics via Graph Neural Networks](https://mlsanity.com/paper/1909.03496/) (2019) - [code2vec: Learning Distributed Representations of Code](https://mlsanity.com/paper/1803.09473/) (2018) - [Learning to Represent Programs with Graphs](https://mlsanity.com/paper/1711.00740/) (2017) - [code2seq: Generating Sequences from Structured Representations of Code](https://mlsanity.com/paper/1808.01400/) (2018) - [A Transformer-based Approach for Source Code Summarization](https://mlsanity.com/paper/2005.00653/) (2020) - [Improving Automatic Source Code Summarization via Deep Reinforcement Learning](https://mlsanity.com/paper/1811.07234/) (2018) - [Tree-to-tree Neural Networks for Program Translation](https://mlsanity.com/paper/1802.03691/) (2018) - [Improved Code Summarization via a Graph Neural Network](https://mlsanity.com/paper/2004.02843/) (2020) - [A Neural Model for Generating Natural Language Summaries of Program Subroutines](https://mlsanity.com/paper/1902.01954/) (2019) - [On the Evaluation of Neural Code Summarization](https://mlsanity.com/paper/2107.07112/) (2021) - [Multi-Modal Attention Network Learning for Semantic Source Code Retrieval](https://mlsanity.com/paper/1909.13516/) (2019) - [TreeBERT: A Tree-Based Pre-Trained Model for Programming Language](https://mlsanity.com/paper/2105.12485/) (2021) - [Retrieve and Refine: Exemplar-based Neural Comment Generation](https://mlsanity.com/paper/1910.10419/) (2019) - [CAST: Enhancing Code Summarization with Hierarchical Splitting and Reconstruction of Abstract Syntax Trees](https://mlsanity.com/paper/2108.12987/) (2021) - [A General Path-Based Representation for Predicting Program Properties](https://mlsanity.com/paper/1803.09544/) (2018) - [Code Summarization with Structure-induced Transformer](https://mlsanity.com/paper/2012.14710/) (2020) - [Automatic Source Code Summarization with Extended Tree-LSTM](https://mlsanity.com/paper/1906.08094/) (2019) - [Project-Level Encoding for Neural Source Code Summarization of Subroutines](https://mlsanity.com/paper/2103.11599/) (2021) - [CoCoSum: Contextual Code Summarization with Multi-Relational Graph Neural Network](https://mlsanity.com/paper/2107.01933/) (2021) - [A Multi-Perspective Architecture for Semantic Code Search](https://mlsanity.com/paper/2005.06980/) (2020) - [Improving Code Summarization with Block-wise Abstract Syntax Tree Splitting](https://mlsanity.com/paper/2103.07845/) (2021) ## Built on by (most cited first, 5 of 5) - [Scaling Code-Assisted Chain-of-Thoughts and Instructions for Model Reasoning](https://mlsanity.com/paper/2510.04081/) (2025) - [Agent4cs: A Multi-agent System for Code Summarization in Large Hierarchical Codebases](https://mlsanity.com/paper/2607.01425/) (2026) - [Secure Resource Allocation via Constrained Deep Reinforcement Learning](https://mlsanity.com/paper/2501.11557/) (2025) - [Enhancing LLM-Based Test Generation by Eliminating Covered Code](https://mlsanity.com/paper/2602.21997/) (2026) - [Breaking Validity-Induced Boundaries to Expand Algorithm Search Space: A Two-Stage AST-Based Operator for LLM-Driven Automated Heuristic Evolution](https://mlsanity.com/paper/2604.16420/) (2026) ## Abstract (arXiv) Programming language understanding and representation (a.k.a code representation learning) has always been a hot and challenging task in software engineering. It aims to apply deep learning techniques to produce numerical representations of the source code features while preserving its semantics. These representations can be used for facilitating subsequent code-related tasks. The abstract syntax tree (AST), a fundamental code feature, illustrates the syntactic information of the source code and has been widely used in code representation learning. However, there is still a lack of systematic and quantitative evaluation of how well AST-based code representation facilitates subsequent code-related tasks. In this paper, we first conduct a comprehensive empirical study to explore the effectiveness of the AST-based code representation in facilitating follow-up code-related tasks. To do so, we compare the performance of models trained with code token sequence (Token for short) based code representation and AST-based code representation on three popular types of code-related tasks. Surprisingly, the overall quantitative statistical results demonstrate that models trained with AST-based code representation consistently perform worse across all three tasks compared to models trained with Token-based code representation. Our further quantitative analysis reveals that models trained with AST-based code representation outperform models trained with Token-based code representation in certain subsets of samples across all three tasks. We also conduct comprehensive experiments to evaluate and reveal the impact of the choice of AST parsing/preprocessing/encoding methods on AST-based code representation and subsequent code-related tasks. Our study provides future researchers with detailed guidance on how to select solutions at each stage to fully exploit AST.