# Neural Code Comprehension: A Learnable Representation of Code Semantics

arXiv: https://arxiv.org/abs/1806.07336 · HTML: https://mlsanity.com/paper/1806.07336/

## Key facts

- Cited by 59 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Jun 2018.

## Built on by (most cited first, 30 of 59)

- [CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation](https://mlsanity.com/paper/2102.04664/) (2021)
- [CodeNet: A Large-Scale AI for Code Dataset for Learning a Diversity of Coding Tasks](https://mlsanity.com/paper/2105.12655/) (2021)
- [PalmTree: Learning an Assembly Language Model for Instruction Embedding](https://mlsanity.com/paper/2103.03809/) (2021)
- [CompilerGym: Robust, Performant Compiler Optimization Environments for AI Research](https://mlsanity.com/paper/2109.08267/) (2021)
- [How could Neural Networks understand Programs?](https://mlsanity.com/paper/2105.04297/) (2021)
- [A Survey on Machine Learning Techniques for Source Code Analysis](https://mlsanity.com/paper/2110.09610/) (2021)
- [Deep Learning for Code Intelligence: Survey, Benchmark and Toolkit](https://mlsanity.com/paper/2401.00288/) (2023)
- [TpuGraphs: A Performance Prediction Dataset on Large Tensor Computational Graphs](https://mlsanity.com/paper/2308.13490/) (2023)
- [CODE-MVP: Learning to Represent Source Code from Multiple Views with Contrastive Pre-Training](https://mlsanity.com/paper/2205.02029/) (2022)
- [Learning to Parallelize with OpenMP by Augmented Heterogeneous AST Representation](https://mlsanity.com/paper/2305.05779/) (2023)
- [Cross-Language Binary-Source Code Matching with Intermediate Representations](https://mlsanity.com/paper/2201.07420/) (2022)
- [A Survey on Pretrained Language Models for Neural Code Intelligence](https://mlsanity.com/paper/2212.10079/) (2022)
- [Learning Compiler Pass Orders using Coreset and Normalized Value Prediction](https://mlsanity.com/paper/2301.05104/) (2023)
- [Learning Intermediate Representations using Graph Neural Networks for NUMA and Prefetchers Optimization](https://mlsanity.com/paper/2203.00611/) (2022)
- [MLGOPerf: An ML Guided Inliner to Optimize Performance](https://mlsanity.com/paper/2207.08389/) (2022)
- [Toward Code Generation: A Survey and Lessons from Semantic Parsing](https://mlsanity.com/paper/2105.03317/) (2021)
- [Performance Optimization using Multimodal Modeling and Heterogeneous GNN](https://mlsanity.com/paper/2304.12568/) (2023)
- [Multi-View Graph Representation for Programming Language Processing: An Investigation into Algorithm Detection](https://mlsanity.com/paper/2202.12481/) (2022)
- [Pre-Training Representations of Binary Code Using Contrastive Learning](https://mlsanity.com/paper/2210.05102/) (2022)
- [CompilerDream: Learning a Compiler World Model for General Code Optimization](https://mlsanity.com/paper/2404.16077/) (2024)
- [On the Limitations of Embedding Based Methods for Measuring Functional Correctness for Code Generation](https://mlsanity.com/paper/2405.01580/) (2024)
- [Learning to Make Compiler Optimizations More Effective](https://mlsanity.com/paper/2102.13514/) (2021)
- [RL4ReAl: Reinforcement Learning for Register Allocation](https://mlsanity.com/paper/2204.02013/) (2022)
- [A Survey of Deep Learning Models for Structural Code Understanding](https://mlsanity.com/paper/2205.01293/) (2022)
- [CodeGRAG: Bridging the Gap between Natural Language and Programming Language via Graphical Retrieval Augmented Generation](https://mlsanity.com/paper/2405.02355/) (2024)
- [ObscuraCoder: Powering Efficient Code LM Pre-Training Via Obfuscation Grounding](https://mlsanity.com/paper/2504.00019/) (2025)
- [deGraphCS: Embedding Variable-based Flow Graph for Neural Code Search](https://mlsanity.com/paper/2103.13020/) (2021)
- [LS-CAT: A Large-Scale CUDA AutoTuning Dataset](https://mlsanity.com/paper/2103.14409/) (2021)
- [Learning to Extend Program Graphs to Work-in-Progress Code](https://mlsanity.com/paper/2105.14038/) (2021)
- [Generating GPU Compiler Heuristics using Reinforcement Learning](https://mlsanity.com/paper/2111.12055/) (2021)

## Abstract (arXiv)

With the recent success of embeddings in natural language processing, research has been conducted into applying similar methods to code analysis. Most works attempt to process the code directly or use a syntactic tree representation, treating it like sentences written in a natural language. However, none of the existing methods are sufficient to comprehend program semantics robustly, due to structural features such as function calls, branching, and interchangeable order of statements. In this paper, we propose a novel processing technique to learn code semantics, and apply it to a variety of program analysis tasks. In particular, we stipulate that a robust distributional hypothesis of code applies to both human- and machine-generated programs. Following this hypothesis, we define an embedding space, inst2vec, based on an Intermediate Representation (IR) of the code that is independent of the source programming language. We provide a novel definition of contextual flow for this IR, leveraging both the underlying data- and control-flow of the program. We then analyze the embeddings qualitatively using analogies and clustering, and evaluate the learned representation on three different high-level tasks. We show that even without fine-tuning, a single RNN architecture and fixed inst2vec embeddings outperform specialized approaches for performance prediction (compute device mapping, optimal thread coarsening); and algorithm classification from raw code (104 classes), where we set a new state-of-the-art.