# Learning Program Semantics with Code Representations: An Empirical Study arXiv: https://arxiv.org/abs/2203.11790 ยท HTML: https://mlsanity.com/paper/2203.11790/ ## Key facts - Cited by 6 papers in the map (data through 14 Oct 2026). - Builds on 31 of its 71 references that are in the map. - First posted on arXiv 22 Mar 2022. ## Builds on (most cited first, 30 of 31) - [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017) - [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016) - [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017) - [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016) - [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014) - [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013) - [Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering](https://mlsanity.com/paper/1606.09375/) (2016) - [Teaching Machines to Read and Comprehend](https://mlsanity.com/paper/1506.03340/) (2015) - [CodeSearchNet Challenge: Evaluating the State of Semantic Code Search](https://mlsanity.com/paper/1909.09436/) (2019) - [Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks](https://mlsanity.com/paper/1503.00075/) (2015) - [Devign: Effective Vulnerability Identification by Learning Comprehensive Program Semantics via Graph Neural Networks](https://mlsanity.com/paper/1909.03496/) (2019) - [Unsupervised Translation of Programming Languages](https://mlsanity.com/paper/2006.03511/) (2020) - [code2vec: Learning Distributed Representations of Code](https://mlsanity.com/paper/1803.09473/) (2018) - [Deep Graph Library: A Graph-Centric, Highly-Performant Package for Graph Neural Networks](https://mlsanity.com/paper/1909.01315/) (2019) - [Learning to Represent Programs with Graphs](https://mlsanity.com/paper/1711.00740/) (2017) - [code2seq: Generating Sequences from Structured Representations of Code](https://mlsanity.com/paper/1808.01400/) (2018) - [A Transformer-based Approach for Source Code Summarization](https://mlsanity.com/paper/2005.00653/) (2020) - [Automated Vulnerability Detection in Source Code Using Deep Representation Learning](https://mlsanity.com/paper/1807.04320/) (2018) - [Retrieval-Augmented Generation for Code Summarization via Hybrid GNN](https://mlsanity.com/paper/2006.05405/) (2020) - [Improved Code Summarization via a Graph Neural Network](https://mlsanity.com/paper/2004.02843/) (2020) - [Detecting Code Clones with Graph Neural Networkand Flow-Augmented Abstract Syntax Tree](https://mlsanity.com/paper/2002.08653/) (2020) - [Structured Neural Summarization](https://mlsanity.com/paper/1811.01824/) (2018) - [Typilus: Neural Type Hints](https://mlsanity.com/paper/2004.10657/) (2020) - [Automatic Source Code Summarization with Extended Tree-LSTM](https://mlsanity.com/paper/1906.08094/) (2019) - [ATOM: Commit Message Generation Based on Abstract Syntax Tree and Hybrid Ranking](https://mlsanity.com/paper/1912.02972/) (2019) - [SPI: Automated Identification of Security Patches via Commits](https://mlsanity.com/paper/2105.14565/) (2021) - [TreeCaps: Tree-Based Capsule Networks for Source Code Processing](https://mlsanity.com/paper/2009.09777/) (2020) - [GraphSearchNet: Enhancing GNNs via Capturing Global Dependencies for Semantic Code Search](https://mlsanity.com/paper/2111.02671/) (2021) - [CORE: Automating Review Recommendation for Code Changes](https://mlsanity.com/paper/1912.09652/) (2019) - [LEOPARD: Identifying Vulnerable Code for Vulnerability Assessment through Program Metrics](https://mlsanity.com/paper/1901.11479/) (2019) ## Built on by (most cited first, 6 of 6) - [Abstract Syntax Tree for Programming Language Understanding and Representation: How Far Are We?](https://mlsanity.com/paper/2312.00413/) (2023) - [A Survey of Source Code Representations for Machine Learning-Based Cybersecurity Tasks](https://mlsanity.com/paper/2403.10646/) (2024) - [xASTNN: Improved Code Representations for Industrial Practice](https://mlsanity.com/paper/2303.07104/) (2023) - [Redundancy and Concept Analysis for Code-trained Language Models](https://mlsanity.com/paper/2305.00875/) (2023) - [Beyond Fidelity: Explaining Vulnerability Localization of Learning-based Detectors](https://mlsanity.com/paper/2401.02686/) (2024) - [MAGNET: A Multi-Graph Attentional Network for Code Clone Detection](https://mlsanity.com/paper/2510.24241/) (2025) ## Abstract (arXiv) Program semantics learning is the core and fundamental for various code intelligent tasks e.g., vulnerability detection, clone detection. A considerable amount of existing works propose diverse approaches to learn the program semantics for different tasks and these works have achieved state-of-the-art performance. However, currently, a comprehensive and systematic study on evaluating different program representation techniques across diverse tasks is still missed. From this starting point, in this paper, we conduct an empirical study to evaluate different program representation techniques. Specifically, we categorize current mainstream code representation techniques into four categories i.e., Feature-based, Sequence-based, Tree-based, and Graph-based program representation technique and evaluate its performance on three diverse and popular code intelligent tasks i.e., {Code Classification}, Vulnerability Detection, and Clone Detection on the public released benchmark. We further design three {research questions (RQs)} and conduct a comprehensive analysis to investigate the performance. By the extensive experimental results, we conclude that (1) The graph-based representation is superior to the other selected techniques across these tasks. (2) Compared with the node type information used in tree-based and graph-based representations, the node textual information is more critical to learning the program semantics. (3) Different tasks require the task-specific semantics to achieve their highest performance, however combining various program semantics from different dimensions such as control dependency, data dependency can still produce promising results.