# On Quantum Natural Policy Gradients

arXiv: https://arxiv.org/abs/2401.08307 · HTML: https://mlsanity.com/paper/2401.08307/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Quantum reinforcement learning, one of 15 directions in Deep reinforcement learning methods.
- Builds on 13 of its 31 references that are in the map.
- First posted on arXiv 16 Jan 2024.

## Placed in

- Deep reinforcement learning methods / [Quantum reinforcement learning](https://mlsanity.com/deeprl/quantum-reinforcement-learning/)

## Builds on (most cited first, 13 of 13)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Variational Quantum Circuits for Deep Reinforcement Learning](https://mlsanity.com/paper/1907.00397/) (2019)
- [Quantum Natural Gradient](https://mlsanity.com/paper/1909.02108/) (2019)
- [Parametrized quantum policies for reinforcement learning](https://mlsanity.com/paper/2103.05577/) (2021)
- [Universal Quantum Control through Deep Reinforcement Learning](https://mlsanity.com/paper/1803.01857/) (2018)
- [Quantum Deep Hedging](https://mlsanity.com/paper/2303.16585/) (2023)
- [Quantum Policy Gradient Algorithm with Optimized Action Decoding](https://mlsanity.com/paper/2212.06663/) (2022)
- [Optimal training of variational quantum algorithms without barren plateaus](https://mlsanity.com/paper/2104.14543/) (2021)
- [Policy Gradients using Variational Quantum Circuits](https://mlsanity.com/paper/2203.10591/) (2022)
- [Quantum policy gradient algorithms](https://mlsanity.com/paper/2212.09328/) (2022)
- [Reinforcement Learning and its Connections with Neuroscience and Psychology](https://mlsanity.com/paper/2007.01099/) (2020)
- [Quantum Natural Policy Gradients: Towards Sample-Efficient Reinforcement Learning](https://mlsanity.com/paper/2304.13571/) (2023)

## Abstract (arXiv)

This research delves into the role of the quantum Fisher Information Matrix (FIM) in enhancing the performance of Parameterized Quantum Circuit (PQC)-based reinforcement learning agents. While previous studies have highlighted the effectiveness of PQC-based policies preconditioned with the quantum FIM in contextual bandits, its impact in broader reinforcement learning contexts, such as Markov Decision Processes, is less clear. Through a detailed analysis of L"owner inequalities between quantum and classical FIMs, this study uncovers the nuanced distinctions and implications of using each type of FIM. Our results indicate that a PQC-based agent using the quantum FIM without additional insights typically incurs a larger approximation error and does not guarantee improved performance compared to the classical FIM. Empirical evaluations in classic control benchmarks suggest even though quantum FIM preconditioning outperforms standard gradient ascent, in general it is not superior to classical FIM preconditioning.