# RL4ReAl: Reinforcement Learning for Register Allocation

arXiv: https://arxiv.org/abs/2204.02013 · HTML: https://mlsanity.com/paper/2204.02013/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 74 references that are in the map.
- First posted on arXiv 5 Apr 2022.

## Builds on (most cited first, 18 of 18)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation](https://mlsanity.com/paper/1806.10293/) (2018)
- [Deep Reinforcement Learning for Autonomous Driving: A Survey](https://mlsanity.com/paper/2002.00444/) (2020)
- [code2vec: Learning Distributed Representations of Code](https://mlsanity.com/paper/1803.09473/) (2018)
- [Neural Code Comprehension: A Learnable Representation of Code Semantics](https://mlsanity.com/paper/1806.07336/) (2018)
- [Deep Reinforcement Learning meets Graph Neural Networks: exploring a routing optimization use case](https://mlsanity.com/paper/1910.07421/) (2019)
- [Ithemal: Accurate, Portable and Fast Basic Block Throughput Estimation using Deep Neural Networks](https://mlsanity.com/paper/1808.07412/) (2018)
- [CompilerGym: Robust, Performant Compiler Optimization Environments for AI Research](https://mlsanity.com/paper/2109.08267/) (2021)
- [Graph Colouring Meets Deep Learning: Effective Graph Neural Network Models for Combinatorial Problems](https://mlsanity.com/paper/1903.04598/) (2019)
- [MLGO: a Machine Learning Guided Compiler Optimizations Framework](https://mlsanity.com/paper/2101.04808/) (2021)
- [NeuroVectorizer: End-to-End Vectorization with Deep Reinforcement Learning](https://mlsanity.com/paper/1909.13639/) (2019)
- [Static Neural Compiler Optimization via Deep Reinforcement Learning](https://mlsanity.com/paper/2008.08951/) (2020)
- [Coloring Big Graphs with AlphaGoZero](https://mlsanity.com/paper/1902.10162/) (2019)
- [AutoPhase: Compiler Phase-Ordering for High Level Synthesis with Deep Reinforcement Learning](https://mlsanity.com/paper/1901.04615/) (2019)
- [Combinatorial Register Allocation and Instruction Scheduling](https://mlsanity.com/paper/1804.02452/) (2018)
- [Optimizing Memory Placement using Evolutionary Graph Reinforcement Learning](https://mlsanity.com/paper/2007.07298/) (2020)

## Built on by (most cited first, 4 of 4)

- [Agentic Harness for Real-World Compilers](https://mlsanity.com/paper/2603.20075/) (2026)
- [VeriLocc: End-to-End Cross-Architecture Register Allocation via LLM](https://mlsanity.com/paper/2506.17506/) (2025)
- [CHEHAB RL: Learning to Optimize Fully Homomorphic Encryption Computations](https://mlsanity.com/paper/2601.19367/) (2026)
- [The Next 700 ML-Enabled Compiler Optimizations](https://mlsanity.com/paper/2311.10800/) (2023)

## Abstract (arXiv)

We aim to automate decades of research and experience in register allocation, leveraging machine learning. We tackle this problem by embedding a multi-agent reinforcement learning algorithm within LLVM, training it with the state of the art techniques. We formalize the constraints that precisely define the problem for a given instruction-set architecture, while ensuring that the generated code preserves semantic correctness. We also develop a gRPC based framework providing a modular and efficient compiler interface for training and inference. Our approach is architecture independent: we show experimental results targeting Intel x86 and ARM AArch64. Our results match or out-perform the heavily tuned, production-grade register allocators of LLVM.