# Learning Semantic Program Embeddings with Graph Interval Neural Network

arXiv: https://arxiv.org/abs/2005.09997 · HTML: https://mlsanity.com/paper/2005.09997/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 May 2020.

## Built on by (most cited first, 20 of 20)

- [Show Your Work: Scratchpads for Intermediate Computation with Language Models](https://mlsanity.com/paper/2112.00114/) (2021)
- [CURE: Code-Aware Neural Machine Translation for Automatic Program Repair](https://mlsanity.com/paper/2103.00073/) (2021)
- [Code Generation Tools (Almost) for Free? A Study of Few-Shot, Pre-Trained Language Models on Code](https://mlsanity.com/paper/2206.01335/) (2022)
- [Code Execution with Pre-trained Language Models](https://mlsanity.com/paper/2305.05383/) (2023)
- [A Survey on Machine Learning Techniques for Source Code Analysis](https://mlsanity.com/paper/2110.09610/) (2021)
- [VELVET: a noVel Ensemble Learning approach to automatically locate VulnErable sTatements](https://mlsanity.com/paper/2112.10893/) (2021)
- [Perfect is the enemy of test oracle](https://mlsanity.com/paper/2302.01488/) (2023)
- [A Survey of Deep Learning Models for Structural Code Understanding](https://mlsanity.com/paper/2205.01293/) (2022)
- [A Library for Representing Python Programs as Graphs for Machine Learning](https://mlsanity.com/paper/2208.07461/) (2022)
- [Precise Learning of Source Code Contextual Semantics via Hierarchical Dependence Structure and Graph Attention Networks](https://mlsanity.com/paper/2111.11435/) (2021)
- [CLAWSAT: Towards Both Robust and Accurate Code Models](https://mlsanity.com/paper/2211.11711/) (2022)
- [WheaCha: A Method for Explaining the Predictions of Models of Code](https://mlsanity.com/paper/2102.04625/) (2021)
- [LAMNER: Code Comment Generation Using Character Language Model and Named Entity Recognition](https://mlsanity.com/paper/2204.09654/) (2022)
- [A Novel Feature Representation for Malware Classification](https://mlsanity.com/paper/2210.09580/) (2022)
- [Implant Global and Local Hierarchy Information to Sequence based Code Representation Models](https://mlsanity.com/paper/2303.07826/) (2023)
- [Towards a Neural Debugger for Python](https://mlsanity.com/paper/2603.09951/) (2026)
- [Infrared: A Meta Bug Detector](https://mlsanity.com/paper/2209.08510/) (2022)
- [Judging Adam: Studying the Performance of Optimization Methods on ML4SE Tasks](https://mlsanity.com/paper/2303.03540/) (2023)
- [A Study of Variable-Role-based Feature Enrichment in Neural Models of Code](https://mlsanity.com/paper/2303.04942/) (2023)
- [Graph Neural Networks For Mapping Variables Between Programs -- Extended Version](https://mlsanity.com/paper/2307.13014/) (2023)

## Abstract (arXiv)

Learning distributed representations of source code has been a challenging task for machine learning models. Earlier works treated programs as text so that natural language methods can be readily applied. Unfortunately, such approaches do not capitalize on the rich structural information possessed by source code. Of late, Graph Neural Network (GNN) was proposed to learn embeddings of programs from their graph representations. Due to the homogeneous and expensive message-passing procedure, GNN can suffer from precision issues, especially when dealing with programs rendered into large graphs. In this paper, we present a new graph neural architecture, called Graph Interval Neural Network (GINN), to tackle the weaknesses of the existing GNN. Unlike the standard GNN, GINN generalizes from a curated graph representation obtained through an abstraction method designed to aid models to learn. In particular, GINN focuses exclusively on intervals for mining the feature representation of a program, furthermore, GINN operates on a hierarchy of intervals for scaling the learning to large graphs. We evaluate GINN for two popular downstream applications: variable misuse prediction and method name prediction. Results show in both cases GINN outperforms the state-of-the-art models by a comfortable margin. We have also created a neural bug detector based on GINN to catch null pointer deference bugs in Java code. While learning from the same 9,000 methods extracted from 64 projects, GINN-based bug detector significantly outperforms GNN-based bug detector on 13 unseen test projects. Next, we deploy our trained GINN-based bug detector and Facebook Infer to scan the codebase of 20 highly starred projects on GitHub. Through our manual inspection, we confirm 38 bugs out of 102 warnings raised by GINN-based bug detector compared to 34 bugs out of 129 warnings for Facebook Infer.