# Optimizing Quantum Variational Circuits with Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/2109.03188 · HTML: https://mlsanity.com/paper/2109.03188/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 72 references that are in the map.
- First posted on arXiv 7 Sep 2021.

## Builds on (most cited first, 25 of 25)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [Soft Actor-Critic Algorithms and Applications](https://mlsanity.com/paper/1812.05905/) (2018)
- [Dota 2 with Large Scale Deep Reinforcement Learning](https://mlsanity.com/paper/1912.06680/) (2019)
- [Deep Reinforcement Learning that Matters](https://mlsanity.com/paper/1709.06560/) (2017)
- [Supervised learning with quantum enhanced feature spaces](https://mlsanity.com/paper/1804.11326/) (2018)
- [Barren plateaus in quantum neural network training landscapes](https://mlsanity.com/paper/1803.11173/) (2018)
- [Parameterized quantum circuits as machine learning models](https://mlsanity.com/paper/1906.07682/) (2019)
- [A rigorous and robust quantum speed-up in supervised machine learning](https://mlsanity.com/paper/2010.02174/) (2020)
- [Maximum Entropy RL (Provably) Solves Some Robust RL Problems](https://mlsanity.com/paper/2103.06257/) (2021)
- [Variational Quantum Circuits for Deep Reinforcement Learning](https://mlsanity.com/paper/1907.00397/) (2019)
- [Learning to Walk via Deep Reinforcement Learning](https://mlsanity.com/paper/1812.11103/) (2018)
- [Absence of Barren Plateaus in Quantum Convolutional Neural Networks](https://mlsanity.com/paper/2011.02966/) (2020)
- [Provably efficient machine learning for quantum many-body problems](https://mlsanity.com/paper/2106.12627/) (2021)
- [Effect of barren plateaus on gradient-free optimization](https://mlsanity.com/paper/2011.12245/) (2020)
- [Reinforcement Learning with Quantum Variational Circuits](https://mlsanity.com/paper/2008.07524/) (2020)
- [TensorFlow Quantum: A Software Framework for Quantum Machine Learning](https://mlsanity.com/paper/2003.02989/) (2020)
- [q-means: A quantum algorithm for unsupervised machine learning](https://mlsanity.com/paper/1812.03584/) (2018)
- [Learning to Optimize Variational Quantum Circuits to Solve Combinatorial Problems](https://mlsanity.com/paper/1911.11071/) (2019)
- [Cost Function Dependent Barren Plateaus in Shallow Parametrized Quantum Circuits](https://mlsanity.com/paper/2001.00550/) (2020)
- [Policy Gradient based Quantum Approximate Optimization Algorithm](https://mlsanity.com/paper/2002.01068/) (2020)
- [Noise-Robust End-to-End Quantum Control using Deep Autoregressive Policy Networks](https://mlsanity.com/paper/2012.06701/) (2020)

## Built on by (most cited first, 9 of 9)

- [Practical and efficient quantum circuit synthesis and transpiling with Reinforcement Learning](https://mlsanity.com/paper/2405.13196/) (2024)
- [Reinforcement learning with learned gadgets to tackle hard quantum problems on real hardware](https://mlsanity.com/paper/2411.00230/) (2024)
- [TensorRL-QAS: Reinforcement learning with tensor networks for improved quantum architecture search](https://mlsanity.com/paper/2505.09371/) (2025)
- [Optimizing the non-Clifford-count in unitary synthesis using Reinforcement Learning](https://mlsanity.com/paper/2509.21709/) (2025)
- [An Empirical Review of Optimization Techniques for Quantum Variational Circuits](https://mlsanity.com/paper/2202.01389/) (2022)
- [BenchRL-QAS: Benchmarking reinforcement learning algorithms for quantum architecture search](https://mlsanity.com/paper/2507.12189/) (2025)
- [QFlowNet: Fast, Diverse, and Efficient Unitary Synthesis with Generative Flow Networks](https://mlsanity.com/paper/2603.03045/) (2026)
- [Model-free Distortion Canceling and Control of Quantum Devices](https://mlsanity.com/paper/2407.09877/) (2024)
- [Modeling quantum neural network gradient with reinforcement learning](https://mlsanity.com/paper/2609.31066/) (2026)

## Abstract (arXiv)

Quantum Machine Learning (QML) is considered to be one of the most promising applications of near term quantum devices. However, the optimization of quantum machine learning models presents numerous challenges arising from the imperfections of hardware and the fundamental obstacles in navigating an exponentially scaling Hilbert space. In this work, we evaluate the potential of contemporary methods in deep reinforcement learning to augment gradient based optimization routines in quantum variational circuits. We find that reinforcement learning augmented optimizers consistently outperform gradient descent in noisy environments. All code and pretrained weights are available to replicate the results or deploy the models at: https://github.com/lockwo/rl_qvc_opt.