# GraphCode2Vec: Generic Code Embedding via Lexical and Program Dependence Analyses

arXiv: https://arxiv.org/abs/2112.01218 · HTML: https://mlsanity.com/paper/2112.01218/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Dec 2021.

## Built on by (most cited first, 11 of 11)

- [Exploring Code Analysis: Zero-Shot Insights on Syntax and Semantics with LLMs](https://mlsanity.com/paper/2305.12138/) (2023)
- [Large Language Models (LLMs) for Source Code Analysis: applications, models and datasets](https://mlsanity.com/paper/2503.17502/) (2025)
- [Unveiling Code Pre-Trained Models: Investigating Syntax and Semantics Capacities](https://mlsanity.com/paper/2212.10017/) (2022)
- [CodeArt: Better Code Models by Attention Regularization When Symbols Are Lacking](https://mlsanity.com/paper/2402.11842/) (2024)
- [Learning Program Representations with a Tree-Structured Transformer](https://mlsanity.com/paper/2208.08643/) (2022)
- [A Survey of Source Code Representations for Machine Learning-Based Cybersecurity Tasks](https://mlsanity.com/paper/2403.10646/) (2024)
- [MIXCODE: Enhancing Code Classification by Mixup-Based Data Augmentation](https://mlsanity.com/paper/2210.03003/) (2022)
- [NeuDep: Neural Binary Memory Dependence Analysis](https://mlsanity.com/paper/2210.02853/) (2022)
- [Power Constrained Autotuning using Graph Neural Networks](https://mlsanity.com/paper/2302.11467/) (2023)
- [GenCode: A Generic Data Augmentation Framework for Boosting Deep Learning-Based Code Understanding](https://mlsanity.com/paper/2402.15769/) (2024)
- [An Exploratory Literature Study on Sharing and Energy Use of Language Models for Source Code](https://mlsanity.com/paper/2307.02443/) (2023)

## Abstract (arXiv)

Code embedding is a keystone in the application of machine learning on several Software Engineering (SE) tasks. To effectively support a plethora of SE tasks, the embedding needs to capture program syntax and semantics in a way that is generic. To this end, we propose the first self-supervised pre-training approach (called GraphCode2Vec) which produces task-agnostic embedding of lexical and program dependence features. GraphCode2Vec achieves this via a synergistic combination of code analysis and Graph Neural Networks. GraphCode2Vec is generic, it allows pre-training, and it is applicable to several SE downstream tasks. We evaluate the effectiveness of GraphCode2Vec on four (4) tasks (method name prediction, solution classification, mutation testing and overfitted patch classification), and compare it with four (4) similarly generic code embedding baselines (Code2Seq, Code2Vec, CodeBERT, GraphCodeBERT) and 7 task-specific, learning-based methods. In particular, GraphCode2Vec is more effective than both generic and task-specific learning-based baselines. It is also complementary and comparable to GraphCodeBERT (a larger and more complex model). We also demonstrate through a probing and ablation study that GraphCode2Vec learns lexical and program dependence features and that self-supervised pre-training improves effectiveness.