# Warm-Start Variational Quantum Policy Iteration

arXiv: https://arxiv.org/abs/2404.10546 · HTML: https://mlsanity.com/paper/2404.10546/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Quantum reinforcement learning, one of 15 directions in Deep reinforcement learning methods.
- Builds on 10 of its 34 references that are in the map.
- First posted on arXiv 16 Apr 2024.

## Placed in

- Deep reinforcement learning methods / [Quantum reinforcement learning](https://mlsanity.com/deeprl/quantum-reinforcement-learning/)

## Builds on (most cited first, 10 of 10)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [Variational Quantum Algorithms](https://mlsanity.com/paper/2012.09265/) (2020)
- [Variational Quantum Circuits for Deep Reinforcement Learning](https://mlsanity.com/paper/1907.00397/) (2019)
- [Parametrized quantum policies for reinforcement learning](https://mlsanity.com/paper/2103.05577/) (2021)
- [Quantum reinforcement learning in continuous action space](https://mlsanity.com/paper/2012.10711/) (2020)
- [Quantum Policy Gradient Algorithm with Optimized Action Decoding](https://mlsanity.com/paper/2212.06663/) (2022)
- [Quantum Reinforcement Learning via Policy Iteration](https://mlsanity.com/paper/2203.01889/) (2022)
- [Quantum Natural Policy Gradients: Towards Sample-Efficient Reinforcement Learning](https://mlsanity.com/paper/2304.13571/) (2023)
- [Comprehensive Library of Variational LSE Solvers](https://mlsanity.com/paper/2404.09916/) (2024)

## Built on by (most cited first, 4 of 4)

- [Robustness and Generalization in Quantum Reinforcement Learning via Lipschitz Regularization](https://mlsanity.com/paper/2410.21117/) (2024)
- [Learning Encodings by Maximizing State Distinguishability: Variational Quantum Error Correction](https://mlsanity.com/paper/2506.11552/) (2025)
- [Learning to Concatenate Quantum Codes](https://mlsanity.com/paper/2604.14931/) (2026)
- [Learning Logical Operations for Arbitrary Quantum Error Correction Codes](https://mlsanity.com/paper/2605.28162/) (2026)

## Abstract (arXiv)

Reinforcement learning is a powerful framework aiming to determine optimal behavior in highly complex decision-making scenarios. This objective can be achieved using policy iteration, which requires to solve a typically large linear system of equations. We propose the variational quantum policy iteration (VarQPI) algorithm, realizing this step with a NISQ-compatible quantum-enhanced subroutine. Its scalability is supported by an analysis of the structure of generic reinforcement learning environments, laying the foundation for potential quantum advantage with utility-scale quantum computers. Furthermore, we introduce the warm-start initialization variant (WS-VarQPI) that significantly reduces resource overhead. The algorithm solves a large FrozenLake environment with an underlying 256x256-dimensional linear system, indicating its practical robustness.