# Quantum Natural Policy Gradients: Towards Sample-Efficient Reinforcement Learning

arXiv: https://arxiv.org/abs/2304.13571 · HTML: https://mlsanity.com/paper/2304.13571/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 39 references that are in the map.
- First posted on arXiv 26 Apr 2023.

## Builds on (most cited first, 17 of 17)

- [New insights and perspectives on the natural gradient method](https://mlsanity.com/paper/1412.1193/) (2014)
- [Parameterized quantum circuits as machine learning models](https://mlsanity.com/paper/1906.07682/) (2019)
- [A rigorous and robust quantum speed-up in supervised machine learning](https://mlsanity.com/paper/2010.02174/) (2020)
- [Noise-Induced Barren Plateaus in Variational Quantum Algorithms](https://mlsanity.com/paper/2007.14384/) (2020)
- [Quantum Natural Gradient](https://mlsanity.com/paper/1909.02108/) (2019)
- [Parametrized quantum policies for reinforcement learning](https://mlsanity.com/paper/2103.05577/) (2021)
- [On the Quantum versus Classical Learnability of Discrete Distributions](https://mlsanity.com/paper/2007.14451/) (2020)
- [A Survey on Quantum Reinforcement Learning](https://mlsanity.com/paper/2211.03464/) (2022)
- [Subtleties in the trainability of quantum machine learning models](https://mlsanity.com/paper/2110.14753/) (2021)
- [Quantum Policy Gradient Algorithm with Optimized Action Decoding](https://mlsanity.com/paper/2212.06663/) (2022)
- [The Sample-Complexity of General Reinforcement Learning](https://mlsanity.com/paper/1308.4828/) (2013)
- [Optimal training of variational quantum algorithms without barren plateaus](https://mlsanity.com/paper/2104.14543/) (2021)
- [Quantum policy gradient algorithms](https://mlsanity.com/paper/2212.09328/) (2022)
- [Can Error Mitigation Improve Trainability of Noisy Variational Quantum Algorithms?](https://mlsanity.com/paper/2109.01051/) (2021)
- [Uncovering Instabilities in Variational-Quantum Deep Q-Networks](https://mlsanity.com/paper/2202.05195/) (2022)
- [Incremental Data-Uploading for Full-Quantum Classification](https://mlsanity.com/paper/2205.03057/) (2022)
- [Quantum Speedup of Natural Gradient for Variational Bayes](https://mlsanity.com/paper/2106.05807/) (2021)

## Built on by (most cited first, 6 of 6)

- [Benchmarking Quantum Reinforcement Learning](https://mlsanity.com/paper/2501.15893/) (2025)
- [Qiskit-Torch-Module: Fast Prototyping of Quantum Neural Networks](https://mlsanity.com/paper/2404.06314/) (2024)
- [Warm-Start Variational Quantum Policy Iteration](https://mlsanity.com/paper/2404.10546/) (2024)
- [Guided-SPSA: Simultaneous Perturbation Stochastic Approximation assisted by the Parameter Shift Rule](https://mlsanity.com/paper/2404.15751/) (2024)
- [Robustness and Generalization in Quantum Reinforcement Learning via Lipschitz Regularization](https://mlsanity.com/paper/2410.21117/) (2024)
- [On Quantum Natural Policy Gradients](https://mlsanity.com/paper/2401.08307/) (2024)

## Abstract (arXiv)

Reinforcement learning is a growing field in AI with a lot of potential. Intelligent behavior is learned automatically through trial and error in interaction with the environment. However, this learning process is often costly. Using variational quantum circuits as function approximators potentially can reduce this cost. In order to implement this, we propose the quantum natural policy gradient (QNPG) algorithm -- a second-order gradient-based routine that takes advantage of an efficient approximation of the quantum Fisher information matrix. We experimentally demonstrate that QNPG outperforms first-order based training on Contextual Bandits environments regarding convergence speed and stability and moreover reduces the sample complexity. Furthermore, we provide evidence for the practical feasibility of our approach by training on a 12-qubit hardware device.