# Variational quantum compiling with double Q-learning

arXiv: https://arxiv.org/abs/2103.11611 · HTML: https://mlsanity.com/paper/2103.11611/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 43 references that are in the map.
- First posted on arXiv 22 Mar 2021.

## Builds on (most cited first, 9 of 9)

- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Supervised learning with quantum enhanced feature spaces](https://mlsanity.com/paper/1804.11326/) (2018)
- [Barren plateaus in quantum neural network training landscapes](https://mlsanity.com/paper/1803.11173/) (2018)
- [Designing Neural Network Architectures using Reinforcement Learning](https://mlsanity.com/paper/1611.02167/) (2016)
- [BlockQNN: Efficient Block-wise Neural Network Architecture Generation](https://mlsanity.com/paper/1808.05584/) (2018)
- [Cost Function Dependent Barren Plateaus in Shallow Parametrized Quantum Circuits](https://mlsanity.com/paper/2001.00550/) (2020)
- [Topological Quantum Compiling with Reinforcement Learning](https://mlsanity.com/paper/2004.04743/) (2020)
- [Reinforcement-Learning-Based Variational Quantum Circuits Optimization for Combinatorial Problems](https://mlsanity.com/paper/1911.04574/) (2019)
- [A Software Methodology for Compiling Quantum Programs](https://mlsanity.com/paper/1604.01401/) (2016)

## Built on by (most cited first, 13 of 13)

- [Barren Plateaus in Variational Quantum Computing](https://mlsanity.com/paper/2405.00781/) (2024)
- [Quantum Architecture Search: A Survey](https://mlsanity.com/paper/2406.06210/) (2024)
- [Practical and efficient quantum circuit synthesis and transpiling with Reinforcement Learning](https://mlsanity.com/paper/2405.13196/) (2024)
- [A Primer on Quantum Machine Learning](https://mlsanity.com/paper/2511.15969/) (2025)
- [Quantum Generative Diffusion Model: A Fully Quantum-Mechanical Model for Generating Quantum State Ensemble](https://mlsanity.com/paper/2401.07039/) (2024)
- [Efficient and practical quantum compiler towards multi-qubit systems with deep reinforcement learning](https://mlsanity.com/paper/2204.06904/) (2022)
- [Optimizing the non-Clifford-count in unitary synthesis using Reinforcement Learning](https://mlsanity.com/paper/2509.21709/) (2025)
- [Quantum Compiling with Reinforcement Learning on a Superconducting Processor](https://mlsanity.com/paper/2406.12195/) (2024)
- [Policy Gradient Approach to Compilation of Variational Quantum Circuits](https://mlsanity.com/paper/2111.10227/) (2021)
- [Quantum continual learning of quantum data realizing knowledge backward transfer](https://mlsanity.com/paper/2203.14032/) (2022)
- [Overcoming Dimensional Factorization Limits in Discrete Diffusion Models through Quantum Joint Distribution Learning](https://mlsanity.com/paper/2505.05151/) (2025)
- [Decoding surface codes with deep reinforcement learning and probabilistic policy reuse](https://mlsanity.com/paper/2212.11890/) (2022)
- [Transpiler Autotuning with Predictive Models for Quantum Circuit Optimization](https://mlsanity.com/paper/2607.29145/) (2026)

## Abstract (arXiv)

Quantum compiling aims to construct a quantum circuit V by quantum gates drawn from a native gate alphabet, which is functionally equivalent to the target unitary U. It is a crucial stage for the running of quantum algorithms on noisy intermediate-scale quantum (NISQ) devices. However, the space for structure exploration of quantum circuit is enormous, resulting in the requirement of human expertise, hundreds of experimentations or modifications from existing quantum circuits. In this paper, we propose a variational quantum compiling (VQC) algorithm based on reinforcement learning (RL), in order to automatically design the structure of quantum circuit for VQC with no human intervention. An agent is trained to sequentially select quantum gates from the native gate alphabet and the qubits they act on by double Q-learning with ε-greedy exploration strategy and experience replay. At first, the agent randomly explores a number of quantum circuits with different structures, and then iteratively discovers structures with higher performance on the learning task. Simulation results show that the proposed method can make exact compilations with less quantum gates compared to previous VQC algorithms. It can reduce the errors of quantum algorithms due to decoherence process and gate noise in NISQ devices, and enable quantum algorithms especially for complex algorithms to be executed within coherence time.