# Learning to Make Compiler Optimizations More Effective

arXiv: https://arxiv.org/abs/2102.13514 · HTML: https://mlsanity.com/paper/2102.13514/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 60 references that are in the map.
- First posted on arXiv 24 Feb 2021.

## Builds on (most cited first, 21 of 21)

- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Bag of Tricks for Efficient Text Classification](https://mlsanity.com/paper/1607.01759/) (2016)
- [FastText.zip: Compressing text classification models](https://mlsanity.com/paper/1612.03651/) (2016)
- [code2vec: Learning Distributed Representations of Code](https://mlsanity.com/paper/1803.09473/) (2018)
- [A Survey of Machine Learning for Big Code and Naturalness](https://mlsanity.com/paper/1709.06182/) (2017)
- [Learning to Represent Programs with Graphs](https://mlsanity.com/paper/1711.00740/) (2017)
- [A Convolutional Attention Network for Extreme Summarization of Source Code](https://mlsanity.com/paper/1602.03001/) (2016)
- [Neural Network-based Graph Embedding for Cross-Platform Binary Code Similarity Detection](https://mlsanity.com/paper/1708.06525/) (2017)
- [DeepBugs: A Learning Approach to Name-based Bug Detection](https://mlsanity.com/paper/1805.11683/) (2018)
- [Generative Code Modeling with Graphs](https://mlsanity.com/paper/1805.08490/) (2018)
- [Neural Code Comprehension: A Learnable Representation of Code Semantics](https://mlsanity.com/paper/1806.07336/) (2018)
- [Automated software vulnerability detection with machine learning](https://mlsanity.com/paper/1803.04497/) (2018)
- [A General Path-Based Representation for Predicting Program Properties](https://mlsanity.com/paper/1803.09544/) (2018)
- [Polyglot: Distributed Word Representations for Multilingual NLP](https://mlsanity.com/paper/1307.1662/) (2013)
- [Automated Correction for Syntax Errors in Programming Assignments using Recurrent Neural Networks](https://mlsanity.com/paper/1603.06129/) (2016)
- [Semantic Code Repair using Neuro-Symbolic Transformation Networks](https://mlsanity.com/paper/1710.11054/) (2017)
- [Learning to Repair Software Vulnerabilities with Generative Adversarial Networks](https://mlsanity.com/paper/1805.07475/) (2018)
- [End-to-End Prediction of Buffer Overruns from Raw Source Code via Neural Memory Networks](https://mlsanity.com/paper/1703.02458/) (2017)
- [Do Multi-Sense Embeddings Improve Natural Language Understanding?](https://mlsanity.com/paper/1506.01070/) (2015)
- [Path-Based Function Embedding and its Application to Specification Mining](https://mlsanity.com/paper/1802.07779/) (2018)
- [Code Vectors: Understanding Programs Through Embedded Abstracted Symbolic Traces](https://mlsanity.com/paper/1803.06686/) (2018)

## Built on by (most cited first, 4 of 4)

- [Learning Performance-Improving Code Edits](https://mlsanity.com/paper/2302.07867/) (2023)
- [CompilerGym: Robust, Performant Compiler Optimization Environments for AI Research](https://mlsanity.com/paper/2109.08267/) (2021)
- [Supersonic: Learning to Generate Source Code Optimizations in C/C++](https://mlsanity.com/paper/2309.14846/) (2023)
- [Learning Compiler Pass Orders using Coreset and Normalized Value Prediction](https://mlsanity.com/paper/2301.05104/) (2023)

## Abstract (arXiv)

Because loops execute their body many times, compiler developers place much emphasis on their optimization. Nevertheless, in view of highly diverse source code and hardware, compilers still struggle to produce optimal target code. The sheer number of possible loop optimizations, including their combinations, exacerbates the problem further. Today's compilers use hard-coded heuristics to decide when, whether, and which of a limited set of optimizations to apply. Often, this leads to highly unstable behavior, making the success of compiler optimizations dependent on the precise way a loop has been written. This paper presents LoopLearner, which addresses the problem of compiler instability by predicting which way of writing a loop will lead to efficient compiled code. To this end, we train a neural network to find semantically invariant source-level transformations for loops that help the compiler generate more efficient code. Our model learns to extract useful features from the raw source code and predicts the speedup that a given transformation is likely to yield. We evaluate LoopLearner with 1,895 loops from various performance-relevant benchmarks. Applying the transformations that our model deems most favorable prior to compilation yields an average speedup of 1.14x. When trying the top-3 suggested transformations, the average speedup even increases to 1.29x. Comparing the approach with an exhaustive search through all available code transformations shows that LoopLearner helps to identify the most beneficial transformations in several orders of magnitude less time.