# Policy Gradients using Variational Quantum Circuits

arXiv: https://arxiv.org/abs/2203.10591 · HTML: https://mlsanity.com/paper/2203.10591/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 58 references that are in the map.
- First posted on arXiv 20 Mar 2022.

## Builds on (most cited first, 23 of 23)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model](https://mlsanity.com/paper/1911.08265/) (2019)
- [Deep Reinforcement Learning for Autonomous Driving: A Survey](https://mlsanity.com/paper/2002.00444/) (2020)
- [Variational Quantum Algorithms](https://mlsanity.com/paper/2012.09265/) (2020)
- [The power of quantum neural networks](https://mlsanity.com/paper/2011.00027/) (2020)
- [Power of data in quantum machine learning](https://mlsanity.com/paper/2011.01938/) (2020)
- [The effect of data encoding on the expressive power of variational quantum machine learning models](https://mlsanity.com/paper/2008.08605/) (2020)
- [Supervised quantum machine learning models are kernel methods](https://mlsanity.com/paper/2101.11020/) (2021)
- [Reinforcement learning based recommender systems: A survey](https://mlsanity.com/paper/2101.06286/) (2021)
- [Variational Quantum Circuits for Deep Reinforcement Learning](https://mlsanity.com/paper/1907.00397/) (2019)
- [Quantum Natural Gradient](https://mlsanity.com/paper/1909.02108/) (2019)
- [Universal Statistics of Fisher Information in Deep Neural Networks: Mean Field Approach](https://mlsanity.com/paper/1806.01316/) (2018)
- [Robust data encodings for quantum classifiers](https://mlsanity.com/paper/2003.01695/) (2020)
- [FinRL: A Deep Reinforcement Learning Library for Automated Stock Trading in Quantitative Finance](https://mlsanity.com/paper/2011.09607/) (2020)
- [Quantum-enhanced machine learning](https://mlsanity.com/paper/1610.08251/) (2016)
- [Noisy intermediate-scale quantum (NISQ) algorithms](https://mlsanity.com/paper/2101.08448/) (2021)
- [Reinforcement Learning with Quantum Variational Circuits](https://mlsanity.com/paper/2008.07524/) (2020)
- [Stochastic gradient descent for hybrid quantum-classical optimization](https://mlsanity.com/paper/1910.01155/) (2019)
- [Machine learning & artificial intelligence in the quantum domain](https://mlsanity.com/paper/1709.02779/) (2017)
- [Quantum reinforcement learning in continuous action space](https://mlsanity.com/paper/2012.10711/) (2020)
- [Comprehensive Review of Deep Reinforcement Learning Methods and Applications in Economics](https://mlsanity.com/paper/2004.01509/) (2020)
- [Exponential improvements for quantum-accessible reinforcement learning](https://mlsanity.com/paper/1710.11160/) (2017)
- [Short Quantum Circuits in Reinforcement Learning Policies for the Vehicle Routing Problem](https://mlsanity.com/paper/2109.07498/) (2021)

## Built on by (most cited first, 15 of 15)

- [Quantum policy gradient algorithms](https://mlsanity.com/paper/2212.09328/) (2022)
- [VQC-Based Reinforcement Learning with Data Re-uploading: Performance and Trainability](https://mlsanity.com/paper/2401.11555/) (2024)
- [A Primer on Quantum Machine Learning](https://mlsanity.com/paper/2511.15969/) (2025)
- [Quantum Advantage Actor-Critic for Reinforcement Learning](https://mlsanity.com/paper/2401.07043/) (2024)
- [A Study on Optimization Techniques for Variational Quantum Circuits in Reinforcement Learning](https://mlsanity.com/paper/2405.12354/) (2024)
- [Trainability issues in quantum policy gradients](https://mlsanity.com/paper/2406.09614/) (2024)
- [Quantum framework for Reinforcement Learning: Integrating Markov decision process, quantum arithmetic, and trajectory search](https://mlsanity.com/paper/2412.18208/) (2024)
- [Dissecting Quantum Reinforcement Learning: A Systematic Evaluation of Key Components](https://mlsanity.com/paper/2511.17112/) (2025)
- [Self-Modulating Quantum Fast-Weight Programmers for Efficient Adaptive Sequential Learning](https://mlsanity.com/paper/2606.24933/) (2026)
- [GPA: Grover Policy Agent for Generating Optimal Quantum Sensor Circuits](https://mlsanity.com/paper/2502.13755/) (2025)
- [Scalable Quantum Reinforcement Learning on NISQ Devices with Dynamic-Circuit Qubit Reuse and Grover Optimization](https://mlsanity.com/paper/2509.16002/) (2025)
- [On Quantum Natural Policy Gradients](https://mlsanity.com/paper/2401.08307/) (2024)
- [HCQA: Hybrid Classical-Quantum Agent for Generating Optimal Quantum Sensor Circuits](https://mlsanity.com/paper/2508.21246/) (2025)
- [Formal Verification of Noisy Quantum Reinforcement Learning Policies](https://mlsanity.com/paper/2512.01502/) (2025)
- [Emergent Cooperation in Quantum Multi-Agent Reinforcement Learning Using Communication](https://mlsanity.com/paper/2601.18419/) (2026)

## Abstract (arXiv)

Variational Quantum Circuits are being used as versatile Quantum Machine Learning models. Some empirical results exhibit an advantage in supervised and generative learning tasks. However, when applied to Reinforcement Learning, less is known. In this work, we considered a Variational Quantum Circuit composed of a low-depth hardware-efficient ansatz as the parameterized policy of a Reinforcement Learning agent. We show that an ε-approximation of the policy gradient can be obtained using a logarithmic number of samples concerning the total number of parameters. We empirically verify that such quantum models behave similarly or even outperform typical classical neural networks used in standard benchmarking environments and in quantum control, using only a fraction of the parameters. Moreover, we study the Barren Plateau phenomenon in quantum policy gradients using the Fisher Information Matrix spectrum.