# Efficient and practical quantum compiler towards multi-qubit systems with deep reinforcement learning

arXiv: https://arxiv.org/abs/2204.06904 · HTML: https://mlsanity.com/paper/2204.06904/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 101 references that are in the map.
- First posted on arXiv 14 Apr 2022.

## Builds on (most cited first, 21 of 21)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Hindsight Experience Replay](https://mlsanity.com/paper/1707.01495/) (2017)
- [Supervised learning with quantum enhanced feature spaces](https://mlsanity.com/paper/1804.11326/) (2018)
- [A Brief Survey of Deep Reinforcement Learning](https://mlsanity.com/paper/1708.05866/) (2017)
- [Safe Model-based Reinforcement Learning with Stability Guarantees](https://mlsanity.com/paper/1705.08551/) (2017)
- [Power of data in quantum machine learning](https://mlsanity.com/paper/2011.01938/) (2020)
- [Bridging the Gap Between Value and Policy Based Reinforcement Learning](https://mlsanity.com/paper/1702.08892/) (2017)
- [Learning by Playing - Solving Sparse Reward Tasks from Scratch](https://mlsanity.com/paper/1802.10567/) (2018)
- [Experimental Quantum Generative Adversarial Networks for Image Generation](https://mlsanity.com/paper/2010.06201/) (2020)
- [Quantum Architecture Search via Deep Reinforcement Learning](https://mlsanity.com/paper/2104.07715/) (2021)
- [Universal Quantum Control through Deep Reinforcement Learning](https://mlsanity.com/paper/1803.01857/) (2018)
- [Towards understanding the power of quantum kernels in the NISQ era](https://mlsanity.com/paper/2103.16774/) (2021)
- [Topological Quantum Compiling with Reinforcement Learning](https://mlsanity.com/paper/2004.04743/) (2020)
- [Variational quantum compiling with double Q-learning](https://mlsanity.com/paper/2103.11611/) (2021)
- [Quantum Logic Gate Synthesis as a Markov Decision Process](https://mlsanity.com/paper/1912.12002/) (2019)
- [Compiling quantum circuits to realistic hardware architectures using temporal planners](https://mlsanity.com/paper/1705.08927/) (2017)
- [Comparing and Integrating Constraint Programming and Temporal Planning for Quantum Circuit Compilation](https://mlsanity.com/paper/1803.06775/) (2018)
- [Quantum compiling with a variational instruction set for accurate and fast quantum computing](https://mlsanity.com/paper/2203.15574/) (2022)
- [Efficient Universal Quantum Compilation: An Inverse-free Solovay-Kitaev Algorithm](https://mlsanity.com/paper/2112.02040/) (2021)
- [Weighted Quantum Channel Compiling through Proximal Policy Optimization](https://mlsanity.com/paper/2111.02426/) (2021)

## Built on by (most cited first, 6 of 6)

- [Differentiable Quantum Architecture Search in Asynchronous Quantum Reinforcement Learning](https://mlsanity.com/paper/2407.18202/) (2024)
- [Quantum Machine Learning Architecture Search via Deep Reinforcement Learning](https://mlsanity.com/paper/2407.20147/) (2024)
- [Quantum Compiling with Reinforcement Learning on a Superconducting Processor](https://mlsanity.com/paper/2406.12195/) (2024)
- [Decoding surface codes with deep reinforcement learning and probabilistic policy reuse](https://mlsanity.com/paper/2212.11890/) (2022)
- [Hybrid Reward-Driven Reinforcement Learning for Efficient Quantum Circuit Synthesis](https://mlsanity.com/paper/2507.16641/) (2025)
- [The DeepXube Software Package for Solving Pathfinding Problems with Learned Heuristic Functions and Search](https://mlsanity.com/paper/2603.23873/) (2026)

## Abstract (arXiv)

Efficient quantum compiling tactics greatly enhance the capability of quantum computers to execute complicated quantum algorithms. Due to its fundamental importance, a plethora of quantum compilers has been designed in past years. However, there are several caveats to current protocols, which are low optimality, high inference time, limited scalability, and lack of universality. To compensate for these defects, here we devise an efficient and practical quantum compiler assisted by advanced deep reinforcement learning (RL) techniques, i.e., data generation, deep Q-learning, and AQ* search. In this way, our protocol is compatible with various quantum machines and can be used to compile multi-qubit operators. We systematically evaluate the performance of our proposal in compiling quantum operators with both inverse-closed and inverse-free universal basis sets. In the task of single-qubit operator compiling, our proposal outperforms other RL-based quantum compilers in the measure of compiling sequence length and inference time. Meanwhile, the output solution is near-optimal, guaranteed by the Solovay-Kitaev theorem. Notably, for the inverse-free universal basis set, the achieved sequence length complexity is comparable with the inverse-based setting and dramatically advances previous methods. These empirical results contribute to improving the inverse-free Solovay-Kitaev theorem. In addition, for the first time, we demonstrate how to leverage RL-based quantum compilers to accomplish two-qubit operator compiling. The achieved results open an avenue for integrating RL with quantum compiling to unify efficiency and practicality and thus facilitate the exploration of quantum advantages.