# Policy Gradient Approach to Compilation of Variational Quantum Circuits

arXiv: https://arxiv.org/abs/2111.10227 · HTML: https://mlsanity.com/paper/2111.10227/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 56 references that are in the map.
- First posted on arXiv 19 Nov 2021.

## Builds on (most cited first, 11 of 11)

- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Barren plateaus in quantum neural network training landscapes](https://mlsanity.com/paper/1803.11173/) (2018)
- [Noise-Induced Barren Plateaus in Variational Quantum Algorithms](https://mlsanity.com/paper/2007.14384/) (2020)
- [Layerwise learning for quantum neural networks](https://mlsanity.com/paper/2006.14904/) (2020)
- [Effect of barren plateaus on gradient-free optimization](https://mlsanity.com/paper/2011.12245/) (2020)
- [Optimizing Quantum Error Correction Codes with Reinforcement Learning](https://mlsanity.com/paper/1812.08451/) (2018)
- [Cost Function Dependent Barren Plateaus in Shallow Parametrized Quantum Circuits](https://mlsanity.com/paper/2001.00550/) (2020)
- [Policy Gradient based Quantum Approximate Optimization Algorithm](https://mlsanity.com/paper/2002.01068/) (2020)
- [Variational quantum compiling with double Q-learning](https://mlsanity.com/paper/2103.11611/) (2021)
- [Taking gradients through experiments: LSTMs and memory proximal policy optimization for black-box quantum control](https://mlsanity.com/paper/1802.04063/) (2018)
- [Quantum Observables for continuous control of the Quantum Approximate Optimization Algorithm via Reinforcement Learning](https://mlsanity.com/paper/1911.09682/) (2019)

## Built on by (most cited first, 3 of 3)

- [Quantum Architecture Search via Continual Reinforcement Learning](https://mlsanity.com/paper/2112.05779/) (2021)
- [Quantum Compiling with Reinforcement Learning on a Superconducting Processor](https://mlsanity.com/paper/2406.12195/) (2024)
- [Auxiliary Task-based Deep Reinforcement Learning for Quantum Control](https://mlsanity.com/paper/2302.14312/) (2023)

## Abstract (arXiv)

We propose a method for finding approximate compilations of quantum unitary transformations, based on techniques from policy gradient reinforcement learning. The choice of a stochastic policy allows us to rephrase the optimization problem in terms of probability distributions, rather than variational gates. In this framework, the optimal configuration is found by optimizing over distribution parameters, rather than over free angles. We show numerically that this approach can be more competitive than gradient-free methods, for a comparable amount of resources, both for noiseless and noisy circuits. Another interesting feature of this approach to variational compilation is that it does not need a separate register and long-range interactions to estimate the end-point fidelity, which is an improvement over methods which rely on the Hilbert-Schmidt test. We expect these techniques to be relevant for training variational circuits in other contexts.