# Generating GPU Compiler Heuristics using Reinforcement Learning

arXiv: https://arxiv.org/abs/2111.12055 · HTML: https://mlsanity.com/paper/2111.12055/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 32 references that are in the map.
- First posted on arXiv 23 Nov 2021.

## Builds on (most cited first, 15 of 15)

- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Conservative Q-Learning for Offline Reinforcement Learning](https://mlsanity.com/paper/2006.04779/) (2020)
- [Understanding deep learning requires rethinking generalization](https://mlsanity.com/paper/1611.03530/) (2016)
- [Learning from Noisy Labels with Deep Neural Networks: A Survey](https://mlsanity.com/paper/2007.08199/) (2020)
- [code2vec: Learning Distributed Representations of Code](https://mlsanity.com/paper/1803.09473/) (2018)
- [Device Placement Optimization with Reinforcement Learning](https://mlsanity.com/paper/1706.04972/) (2017)
- [DisCor: Corrective Feedback in Reinforcement Learning via Distribution Correction](https://mlsanity.com/paper/2003.07305/) (2020)
- [Neural Code Comprehension: A Learnable Representation of Code Semantics](https://mlsanity.com/paper/1806.07336/) (2018)
- [A Survey on Compiler Autotuning using Machine Learning](https://mlsanity.com/paper/1801.04405/) (2018)
- [MLGO: a Machine Learning Guided Compiler Optimizations Framework](https://mlsanity.com/paper/2101.04808/) (2021)
- [NeuroVectorizer: End-to-End Vectorization with Deep Reinforcement Learning](https://mlsanity.com/paper/1909.13639/) (2019)
- [Deep Reinforcement Learning for Imbalanced Classification](https://mlsanity.com/paper/1901.01379/) (2019)
- [ProGraML: Graph-based Deep Learning for Program Optimization and Analysis](https://mlsanity.com/paper/2003.10536/) (2020)
- [AutoPhase: Compiler Phase-Ordering for High Level Synthesis with Deep Reinforcement Learning](https://mlsanity.com/paper/1901.04615/) (2019)
- [RL-Scope: Cross-Stack Profiling for Deep Reinforcement Learning Workloads](https://mlsanity.com/paper/2102.04285/) (2021)

## Built on by (most cited first, 3 of 3)

- [Discovering Software Parallelization Points Using Deep Neural Networks](https://mlsanity.com/paper/2509.16215/) (2025)
- [Learning to generate Reliable Broadcast Algorithms](https://mlsanity.com/paper/2208.00525/) (2022)
- [Path Planning using Reinforcement Learning: A Policy Iteration Approach](https://mlsanity.com/paper/2303.07535/) (2023)

## Abstract (arXiv)

GPU compilers are complex software programs with many optimizations specific to target hardware. These optimizations are often controlled by heuristics hand-designed by compiler experts using time- and resource-intensive processes. In this paper, we developed a GPU compiler autotuning framework that uses off-policy deep reinforcement learning to generate heuristics that improve the frame rates of graphics applications. Furthermore, we demonstrate the resilience of these learned heuristics to frequent compiler updates by analyzing their stability across a year of code check-ins without retraining. We show that our machine learning-based compiler autotuning framework matches or surpasses the frame rates for 98% of graphics benchmarks with an average uplift of 1.6% up to 15.8%.