# Practical and efficient quantum circuit synthesis and transpiling with Reinforcement Learning

arXiv: https://arxiv.org/abs/2405.13196 · HTML: https://mlsanity.com/paper/2405.13196/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Placed in Quantum reinforcement learning, one of 15 directions in Deep reinforcement learning methods.
- Builds on 12 of its 53 references that are in the map.
- First posted on arXiv 21 May 2024.

## Placed in

- Deep reinforcement learning methods / [Quantum reinforcement learning](https://mlsanity.com/deeprl/quantum-reinforcement-learning/)

## Builds on (most cited first, 12 of 12)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [Curriculum Learning for Reinforcement Learning Domains: A Framework and Survey](https://mlsanity.com/paper/2003.04960/) (2020)
- [Molecular Transformer - A Model for Uncertainty-Calibrated Chemical Reaction Prediction](https://mlsanity.com/paper/1811.02633/) (2018)
- [A Thorough Examination of Decoding Methods in the Era of LLMs](https://mlsanity.com/paper/2402.06925/) (2024)
- [Quantum Circuit Optimization with AlphaTensor](https://mlsanity.com/paper/2402.14396/) (2024)
- [Variational quantum compiling with double Q-learning](https://mlsanity.com/paper/2103.11611/) (2021)
- [Compiler Optimization for Quantum Computing Using Reinforcement Learning](https://mlsanity.com/paper/2212.04508/) (2022)
- [Optimizing Quantum Variational Circuits with Deep Reinforcement Learning](https://mlsanity.com/paper/2109.03188/) (2021)
- [Optimal qubit assignment and routing via integer programming](https://mlsanity.com/paper/2106.06446/) (2021)
- [Improving Quantum Computation by Optimized Qubit Routing](https://mlsanity.com/paper/2206.01294/) (2022)
- [Towards a SAT Encoding for Quantum Circuits: A Journey From Classical Circuits to Clifford Circuits and Beyond](https://mlsanity.com/paper/2203.00698/) (2022)

## Built on by (most cited first, 22 of 22)

- [Qiskit Code Assistant: Training LLMs for generating Quantum Computing Code](https://mlsanity.com/paper/2405.19495/) (2024)
- [Optimizing the non-Clifford-count in unitary synthesis using Reinforcement Learning](https://mlsanity.com/paper/2509.21709/) (2025)
- [QNAS: A Neural Architecture Search Framework for Accurate and Efficient Quantum Neural Networks](https://mlsanity.com/paper/2604.07013/) (2026)
- [Quantum Artificial Intelligence: A Brief Survey](https://mlsanity.com/paper/2408.10726/) (2024)
- [Pauli Network Circuit Synthesis with Reinforcement Learning](https://mlsanity.com/paper/2503.14448/) (2025)
- [Beyond Reinforcement Learning: Fast and Scalable Quantum Circuit Synthesis](https://mlsanity.com/paper/2602.15146/) (2026)
- [AlphaCNOT: Learning CNOT Minimization with Model-Based Planning](https://mlsanity.com/paper/2604.13812/) (2026)
- [Attention-Based Deep Reinforcement Learning for Qubit Allocation in Modular Quantum Architectures](https://mlsanity.com/paper/2406.11452/) (2024)
- [CNOT Minimal Circuit Synthesis: A Reinforcement Learning Approach](https://mlsanity.com/paper/2510.23304/) (2025)
- [Replay-buffer engineering for noise-aware quantum circuit optimization](https://mlsanity.com/paper/2604.21863/) (2026)
- [Aligning Quantum Operators with Large Language Models](https://mlsanity.com/paper/2606.13811/) (2026)
- [When Close Enough Is Not Enough: Autoregressive Drift in Quantum Circuit Synthesis](https://mlsanity.com/paper/2607.12780/) (2026)
- [AI methods for approximate compiling of unitaries](https://mlsanity.com/paper/2407.21225/) (2024)
- [AI Methods for Permutation Circuit Synthesis Across Generic Topologies](https://mlsanity.com/paper/2509.16020/) (2025)
- [QAP-Router: Tackling Qubit Routing as Dynamic Quadratic Assignment with Reinforcement Learning](https://mlsanity.com/paper/2605.12365/) (2026)
- [CO-MAP: A Reinforcement Learning Approach to the Qubit Allocation Problem](https://mlsanity.com/paper/2605.13638/) (2026)
- [When AI meets quantum information: A comprehensive review](https://mlsanity.com/paper/2607.00365/) (2026)
- [Transpiler Autotuning with Predictive Models for Quantum Circuit Optimization](https://mlsanity.com/paper/2607.29145/) (2026)
- [AutoQuREO: A Framework for Automated Quantum Resource Estimation and Optimization](https://mlsanity.com/paper/2608.12936/) (2026)
- [AlphaClifford: Efficient Clifford Synthesis and Transpilation with Model-based RL](https://mlsanity.com/paper/2608.18946/) (2026)
- [Predicting Resource Efficient Hamiltonian Decomposition for Continuous-Time Quantum Walk Simulations](https://mlsanity.com/paper/2608.20660/) (2026)
- [Generative Replay Mitigates Sample Starvation in Quantum Architecture Search](https://mlsanity.com/paper/2609.11248/) (2026)

## Abstract (arXiv)

This paper demonstrates the integration of Reinforcement Learning (RL) into quantum transpiling workflows, significantly enhancing the synthesis and routing of quantum circuits. By employing RL, we achieve near-optimal synthesis of Linear Function, Clifford, and Permutation circuits, up to 9, 11 and 65 qubits respectively, while being compatible with native device instruction sets and connectivity constraints, and orders of magnitude faster than optimization methods such as SAT solvers. We also achieve significant reductions in two-qubit gate depth and count for circuit routing up to 133 qubits with respect to other routing heuristics such as SABRE. We find the method to be efficient enough to be useful in practice in typical quantum transpiling pipelines. Our results set the stage for further AI-powered enhancements of quantum computing workflows.