# Scalable Quantum Reinforcement Learning on NISQ Devices with Dynamic-Circuit Qubit Reuse and Grover Optimization

arXiv: https://arxiv.org/abs/2509.16002 · HTML: https://mlsanity.com/paper/2509.16002/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Quantum reinforcement learning, one of 15 directions in Deep reinforcement learning methods.
- Builds on 17 of its 53 references that are in the map.
- First posted on arXiv 19 Sep 2025.

## Placed in

- Deep reinforcement learning methods / [Quantum reinforcement learning](https://mlsanity.com/deeprl/quantum-reinforcement-learning/)

## Builds on (most cited first, 17 of 17)

- [Safe, Multi-Agent, Reinforcement Learning for Autonomous Driving](https://mlsanity.com/paper/1610.03295/) (2016)
- [Variational Quantum Circuits for Deep Reinforcement Learning](https://mlsanity.com/paper/1907.00397/) (2019)
- [Parametrized quantum policies for reinforcement learning](https://mlsanity.com/paper/2103.05577/) (2021)
- [Reinforcement Learning with Quantum Variational Circuits](https://mlsanity.com/paper/2008.07524/) (2020)
- [Quantum reinforcement learning in continuous action space](https://mlsanity.com/paper/2012.10711/) (2020)
- [Quantum Algorithms for Reinforcement Learning with a Generative Model](https://mlsanity.com/paper/2112.08451/) (2021)
- [Introduction to Quantum Reinforcement Learning: Theory and PennyLane-based Implementation](https://mlsanity.com/paper/2108.06849/) (2021)
- [Quantum Policy Iteration via Amplitude Estimation and Grover Search -- Towards Quantum Advantage for Reinforcement Learning](https://mlsanity.com/paper/2206.04741/) (2022)
- [Deep Reinforcement Learning with Quantum-inspired Experience Replay](https://mlsanity.com/paper/2101.02034/) (2021)
- [Quantum Reinforcement Learning via Policy Iteration](https://mlsanity.com/paper/2203.01889/) (2022)
- [Variational Quantum Soft Actor-Critic](https://mlsanity.com/paper/2112.11921/) (2021)
- [Policy Gradients using Variational Quantum Circuits](https://mlsanity.com/paper/2203.10591/) (2022)
- [IBM Quantum Computers: Evolution, Performance, and Future Directions](https://mlsanity.com/paper/2410.00916/) (2024)
- [Intelligent Trajectory Planning in UAV-mounted Wireless Networks: A Quantum-Inspired Reinforcement Learning Perspective](https://mlsanity.com/paper/2007.13418/) (2020)
- [Quantum bandit with amplitude amplification exploration in an adversarial environment](https://mlsanity.com/paper/2208.07144/) (2022)
- [Quantum framework for Reinforcement Learning: Integrating Markov decision process, quantum arithmetic, and trajectory search](https://mlsanity.com/paper/2412.18208/) (2024)
- [QDB: From Quantum Algorithms Towards Correct Quantum Programs](https://mlsanity.com/paper/1811.05447/) (2018)

## Built on by (most cited first, 1 of 1)

- [Do Quantum AIs Dream in Paths? Path-Integral Slow Thinking through Grover Interference](https://mlsanity.com/paper/2609.05842/) (2026)

## Abstract (arXiv)

A scalable and resource-efficient quantum reinforcement learning framework is presented that eliminates the linear qubit-scaling barrier in multi-step quantum Markov decision processes (QMDPs). The proposed framework integrates a QMDP formulation, dynamic-circuit execution, and Grover-based amplitude amplification into a unified quantum-native architecture. Environment dynamics are encoded entirely within quantum Hilbert space, enabling coherent superposition over state-action sequences and a direct quantum agent-environment interface without intermediate quantum-to-classical conversion. The central contribution is a dynamic execution model for multi-step QMDPs that employs mid-circuit measurement and reset to recycle a fixed physical quantum register across sequential interactions. This approach preserves trajectory fidelity relative to a static unrolled QMDP, generating identical state-action sequences while reducing the physical qubit requirement from 7xT to a constant 7, independent of the interaction horizon T. Thus, the qubit complexity of multi-step QMDPs is transformed from O(T) to O(1) while maintaining functional equivalence at the level of trajectory generation. Trajectory returns are evaluated via quantum arithmetic, and high-return trajectories are marked and amplified using amplitude amplification to increase their sampling probability. Simulations confirm preservation of trajectory fidelity with a 66% qubit reduction compared to a static design. Experimental execution on an IBM Heron-class processor demonstrates feasibility on noisy intermediate-scale quantum hardware, establishing a scalable and resource-efficient foundation for large-scale quantum-native reinforcement learning.