# ProGraML: Graph-based Deep Learning for Program Optimization and Analysis

arXiv: https://arxiv.org/abs/2003.10536 · HTML: https://mlsanity.com/paper/2003.10536/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Mar 2020.

## Built on by (most cited first, 23 of 23)

- [Proof Artifact Co-training for Theorem Proving with Language Models](https://mlsanity.com/paper/2102.06203/) (2021)
- [Transformers Meet Directed Graphs](https://mlsanity.com/paper/2302.00049/) (2023)
- [How could Neural Networks understand Programs?](https://mlsanity.com/paper/2105.04297/) (2021)
- [TpuGraphs: A Performance Prediction Dataset on Large Tensor Computational Graphs](https://mlsanity.com/paper/2308.13490/) (2023)
- [Supersonic: Learning to Generate Source Code Optimizations in C/C++](https://mlsanity.com/paper/2309.14846/) (2023)
- [CODE-MVP: Learning to Represent Source Code from Multiple Views with Contrastive Pre-Training](https://mlsanity.com/paper/2205.02029/) (2022)
- [Understanding Programs by Exploiting (Fuzzing) Test Cases](https://mlsanity.com/paper/2305.13592/) (2023)
- [Learning C to x86 Translation: An Experiment in Neural Compilation](https://mlsanity.com/paper/2108.07639/) (2021)
- [Learning to Parallelize in a Shared-Memory Environment with Transformers](https://mlsanity.com/paper/2204.12835/) (2022)
- [Learning Intermediate Representations using Graph Neural Networks for NUMA and Prefetchers Optimization](https://mlsanity.com/paper/2203.00611/) (2022)
- [GRANITE: A Graph Neural Network Model for Basic Block Throughput Estimation](https://mlsanity.com/paper/2210.03894/) (2022)
- [A Reinforcement Learning Environment for Polyhedral Optimizations](https://mlsanity.com/paper/2104.13732/) (2021)
- [LS-CAT: A Large-Scale CUDA AutoTuning Dataset](https://mlsanity.com/paper/2103.14409/) (2021)
- [Learning to Extend Program Graphs to Work-in-Progress Code](https://mlsanity.com/paper/2105.14038/) (2021)
- [Generating GPU Compiler Heuristics using Reinforcement Learning](https://mlsanity.com/paper/2111.12055/) (2021)
- [PERFOGRAPH: A Numerical Aware Program Graph Representation for Performance Optimization and Program Analysis](https://mlsanity.com/paper/2306.00210/) (2023)
- [Federated Data Science to Break Down Silos [Vision]](https://mlsanity.com/paper/2111.13186/) (2021)
- [Tadashi: Enabling AI-Based Automated Code Generation With Guaranteed Correctness](https://mlsanity.com/paper/2410.03210/) (2024)
- [Call Me Maybe: Enhancing JavaScript Call Graph Construction using Graph Neural Networks](https://mlsanity.com/paper/2506.18191/) (2025)
- [ROAM: memory-efficient large DNN training via optimized operator ordering and memory layout](https://mlsanity.com/paper/2310.19295/) (2023)
- [Analysing the Behaviour of Tree-Based Neural Networks in Regression Tasks](https://mlsanity.com/paper/2406.11437/) (2024)
- [A Deep Learning Model for Predicting Transformation Legality](https://mlsanity.com/paper/2511.06120/) (2025)
- [Learning Reasoning World Models for Parallel Code](https://mlsanity.com/paper/2604.20926/) (2026)

## Abstract (arXiv)

The increasing complexity of computing systems places a tremendous burden on optimizing compilers, requiring ever more accurate and aggressive optimizations. Machine learning offers significant benefits for constructing optimization heuristics but there remains a gap between what state-of-the-art methods achieve and the performance of an optimal heuristic. Closing this gap requires improvements in two key areas: a representation that accurately captures the semantics of programs, and a model architecture with sufficient expressiveness to reason about this representation. We introduce ProGraML - Program Graphs for Machine Learning - a novel graph-based program representation using a low level, language agnostic, and portable format; and machine learning models capable of performing complex downstream tasks over these graphs. The ProGraML representation is a directed attributed multigraph that captures control, data, and call relations, and summarizes instruction and operand types and ordering. Message Passing Neural Networks propagate information through this structured representation, enabling whole-program or per-vertex classification tasks. ProGraML provides a general-purpose program representation that equips learnable models to perform the types of program analysis that are fundamental to optimization. To this end, we evaluate the performance of our approach first on a suite of traditional compiler analysis tasks: control flow reachability, dominator trees, data dependencies, variable liveness, and common subexpression detection. On a benchmark dataset of 250k LLVM-IR files covering six source programming languages, ProGraML achieves an average 94.0 F1 score, significantly outperforming the state-of-the-art approaches. We then apply our approach to two high-level tasks - heterogeneous device mapping and program classification - setting new state-of-the-art performance in both.