# Efficient quantum recurrent reinforcement learning via quantum reservoir computing

arXiv: https://arxiv.org/abs/2309.07339 · HTML: https://mlsanity.com/paper/2309.07339/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 43 references that are in the map.
- First posted on arXiv 13 Sep 2023.

## Builds on (most cited first, 20 of 20)

- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [The power of quantum neural networks](https://mlsanity.com/paper/2011.00027/) (2020)
- [Generalization in quantum machine learning from few training data](https://mlsanity.com/paper/2111.05292/) (2021)
- [Variational Quantum Circuits for Deep Reinforcement Learning](https://mlsanity.com/paper/1907.00397/) (2019)
- [Quantum Long Short-Term Memory](https://mlsanity.com/paper/2009.01783/) (2020)
- [Recurrent Quantum Neural Networks](https://mlsanity.com/paper/2006.14619/) (2020)
- [Reinforcement Learning with Quantum Variational Circuits](https://mlsanity.com/paper/2008.07524/) (2020)
- [The Dawn of Quantum Natural Language Processing](https://mlsanity.com/paper/2110.06510/) (2021)
- [Decentralizing Feature Extraction with Quantum Convolutional Neural Network for Automatic Speech Recognition](https://mlsanity.com/paper/2010.13309/) (2020)
- [An end-to-end trainable hybrid classical-quantum classifier](https://mlsanity.com/paper/2102.02416/) (2021)
- [Variational Quantum Reinforcement Learning via Evolutionary Optimization](https://mlsanity.com/paper/2109.00540/) (2021)
- [When BERT Meets Quantum Temporal Convolution Learning for Text Classification in Heterogeneous Computing](https://mlsanity.com/paper/2203.03550/) (2022)
- [Theoretical Error Performance Analysis for Variational Quantum Circuit Based Functional Regression](https://mlsanity.com/paper/2206.04804/) (2022)
- [PQLM -- Multilingual Decentralized Portable Quantum Language Model for Privacy Protection](https://mlsanity.com/paper/2210.03221/) (2022)
- [Quantum Multi-Agent Reinforcement Learning via Variational Quantum Circuit Design](https://mlsanity.com/paper/2203.10443/) (2022)
- [Quantum deep recurrent reinforcement learning](https://mlsanity.com/paper/2210.14876/) (2022)
- [Asynchronous training of quantum reinforcement learning](https://mlsanity.com/paper/2301.05096/) (2023)
- [Variational Quantum Soft Actor-Critic](https://mlsanity.com/paper/2112.11921/) (2021)
- [Reservoir Computing via Quantum Recurrent Neural Networks](https://mlsanity.com/paper/2211.02612/) (2022)
- [Quantum deep Q learning with distributed prioritized experience replay](https://mlsanity.com/paper/2304.09648/) (2023)

## Built on by (most cited first, 14 of 14)

- [Learning to Program Variational Quantum Circuits with Fast Weights](https://mlsanity.com/paper/2402.17760/) (2024)
- [Differentiable Quantum Architecture Search in Asynchronous Quantum Reinforcement Learning](https://mlsanity.com/paper/2407.18202/) (2024)
- [Quantum Machine Learning Architecture Search via Deep Reinforcement Learning](https://mlsanity.com/paper/2407.20147/) (2024)
- [Evolutionary Optimization for Designing Variational Quantum Circuits with High Model Capacity](https://mlsanity.com/paper/2412.12484/) (2024)
- [An Introduction to Quantum Reinforcement Learning (QRL)](https://mlsanity.com/paper/2409.05846/) (2024)
- [Differentiable Quantum Architecture Search in Quantum-Enhanced Neural Network Parameter Generation](https://mlsanity.com/paper/2505.09653/) (2025)
- [Learning to Program Quantum Measurements for Machine Learning](https://mlsanity.com/paper/2505.13525/) (2025)
- [Quantum Long Short-term Memory with Differentiable Architecture Search](https://mlsanity.com/paper/2508.14955/) (2025)
- [Gated QKAN-FWP: Scalable Quantum-inspired Sequence Learning](https://mlsanity.com/paper/2605.06734/) (2026)
- [Introduction to Quantum Machine Learning and Quantum Architecture Search](https://mlsanity.com/paper/2504.16131/) (2025)
- [Parameter-Efficient Quantum-Inspired Fast Weight Programmers for Traffic-Matrix Forecasting](https://mlsanity.com/paper/2606.27821/) (2026)
- [Self-Modulating Quantum Fast-Weight Programmers for Efficient Adaptive Sequential Learning](https://mlsanity.com/paper/2606.24933/) (2026)
- [QnRL: Quantum-Native Reinforcement Learning](https://mlsanity.com/paper/2606.08276/) (2026)
- [Quantum Reservoir Computing: Recent Advances and Future Directions](https://mlsanity.com/paper/2607.18552/) (2026)

## Abstract (arXiv)

Quantum reinforcement learning (QRL) has emerged as a framework to solve sequential decision-making tasks, showcasing empirical quantum advantages. A notable development is through quantum recurrent neural networks (QRNNs) for memory-intensive tasks such as partially observable environments. However, QRL models incorporating QRNN encounter challenges such as inefficient training of QRL with QRNN, given that the computation of gradients in QRNN is both computationally expensive and time-consuming. This work presents a novel approach to address this challenge by constructing QRL agents utilizing QRNN-based reservoirs, specifically employing quantum long short-term memory (QLSTM). QLSTM parameters are randomly initialized and fixed without training. The model is trained using the asynchronous advantage actor-aritic (A3C) algorithm. Through numerical simulations, we validate the efficacy of our QLSTM-Reservoir RL framework. Its performance is assessed on standard benchmarks, demonstrating comparable results to a fully trained QLSTM RL model with identical architecture and training settings.