# Asynchronous training of quantum reinforcement learning

arXiv: https://arxiv.org/abs/2301.05096 · HTML: https://mlsanity.com/paper/2301.05096/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 47 references that are in the map.
- First posted on arXiv 12 Jan 2023.

## Builds on (most cited first, 29 of 29)

- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [Variational Quantum Algorithms](https://mlsanity.com/paper/2012.09265/) (2020)
- [Variational Quantum Circuits for Deep Reinforcement Learning](https://mlsanity.com/paper/1907.00397/) (2019)
- [Quantum Long Short-Term Memory](https://mlsanity.com/paper/2009.01783/) (2020)
- [Federated Quantum Machine Learning](https://mlsanity.com/paper/2103.12010/) (2021)
- [Noisy intermediate-scale quantum (NISQ) algorithms](https://mlsanity.com/paper/2101.08448/) (2021)
- [Reinforcement Learning with Quantum Variational Circuits](https://mlsanity.com/paper/2008.07524/) (2020)
- [Quantum Convolutional Neural Networks for High Energy Physics Data Analysis](https://mlsanity.com/paper/2012.12177/) (2020)
- [The Dawn of Quantum Natural Language Processing](https://mlsanity.com/paper/2110.06510/) (2021)
- [Decentralizing Feature Extraction with Quantum Convolutional Neural Network for Automatic Speech Recognition](https://mlsanity.com/paper/2010.13309/) (2020)
- [Quantum Federated Learning with Quantum Data](https://mlsanity.com/paper/2106.00005/) (2021)
- [A Survey on Quantum Reinforcement Learning](https://mlsanity.com/paper/2211.03464/) (2022)
- [An end-to-end trainable hybrid classical-quantum classifier](https://mlsanity.com/paper/2102.02416/) (2021)
- [Variational Quantum Reinforcement Learning via Evolutionary Optimization](https://mlsanity.com/paper/2109.00540/) (2021)
- [When BERT Meets Quantum Temporal Convolution Learning for Text Classification in Heterogeneous Computing](https://mlsanity.com/paper/2203.03550/) (2022)
- [Quantum reinforcement learning in continuous action space](https://mlsanity.com/paper/2012.10711/) (2020)
- [QTN-VQC: An End-to-End Learning framework for Quantum Neural Networks](https://mlsanity.com/paper/2110.03861/) (2021)
- [Quantum Multi-Agent Reinforcement Learning via Variational Quantum Circuit Design](https://mlsanity.com/paper/2203.10443/) (2022)
- [Quantum deep recurrent reinforcement learning](https://mlsanity.com/paper/2210.14876/) (2022)
- [Quantum Multi-Agent Meta Reinforcement Learning](https://mlsanity.com/paper/2208.11510/) (2022)
- [Quantum Policy Iteration via Amplitude Estimation and Grover Search -- Towards Quantum Advantage for Reinforcement Learning](https://mlsanity.com/paper/2206.04741/) (2022)
- [Quantum Policy Gradient Algorithm with Optimized Action Decoding](https://mlsanity.com/paper/2212.06663/) (2022)
- [Quantum Reinforcement Learning via Policy Iteration](https://mlsanity.com/paper/2203.01889/) (2022)
- [Quantum Deep Reinforcement Learning for Robot Navigation Tasks](https://mlsanity.com/paper/2202.12180/) (2022)
- [Variational Quantum Soft Actor-Critic](https://mlsanity.com/paper/2112.11921/) (2021)
- [Quantum policy gradient algorithms](https://mlsanity.com/paper/2212.09328/) (2022)
- [Reservoir Computing via Quantum Recurrent Neural Networks](https://mlsanity.com/paper/2211.02612/) (2022)
- [Variational Quantum Soft Actor-Critic for Robotic Arm Control](https://mlsanity.com/paper/2212.11681/) (2022)

## Built on by (most cited first, 20 of 20)

- [Quantum Architecture Search: A Survey](https://mlsanity.com/paper/2406.06210/) (2024)
- [Learning to Program Variational Quantum Circuits with Fast Weights](https://mlsanity.com/paper/2402.17760/) (2024)
- [Efficient quantum recurrent reinforcement learning via quantum reservoir computing](https://mlsanity.com/paper/2309.07339/) (2023)
- [Differentiable Quantum Architecture Search in Asynchronous Quantum Reinforcement Learning](https://mlsanity.com/paper/2407.18202/) (2024)
- [Quantum Advantage Actor-Critic for Reinforcement Learning](https://mlsanity.com/paper/2401.07043/) (2024)
- [Evolutionary Optimization for Designing Variational Quantum Circuits with High Model Capacity](https://mlsanity.com/paper/2412.12484/) (2024)
- [eQMARL: Entangled Quantum Multi-Agent Reinforcement Learning for Distributed Cooperation over Quantum Channels](https://mlsanity.com/paper/2405.17486/) (2024)
- [An Introduction to Quantum Reinforcement Learning (QRL)](https://mlsanity.com/paper/2409.05846/) (2024)
- [Differentiable Quantum Architecture Search in Quantum-Enhanced Neural Network Parameter Generation](https://mlsanity.com/paper/2505.09653/) (2025)
- [Learning to Program Quantum Measurements for Machine Learning](https://mlsanity.com/paper/2505.13525/) (2025)
- [Quantum Reinforcement Learning by Adaptive Non-local Observables](https://mlsanity.com/paper/2507.19629/) (2025)
- [Machine Learning Applications of Quantum Computing: A Review](https://mlsanity.com/paper/2406.13262/) (2024)
- [Q-Fusion: Diffusing Quantum Circuits](https://mlsanity.com/paper/2504.20794/) (2025)
- [Gated QKAN-FWP: Scalable Quantum-inspired Sequence Learning](https://mlsanity.com/paper/2605.06734/) (2026)
- [Introduction to Quantum Machine Learning and Quantum Architecture Search](https://mlsanity.com/paper/2504.16131/) (2025)
- [Quantum deep Q learning with distributed prioritized experience replay](https://mlsanity.com/paper/2304.09648/) (2023)
- [Quantum Machine Learning: An Interplay Between Quantum Computing and Machine Learning](https://mlsanity.com/paper/2411.09403/) (2024)
- [Quantum Hierarchical Reinforcement Learning via Variational Quantum Circuits](https://mlsanity.com/paper/2605.03434/) (2026)
- [Quantum-Enhanced Forecasting for Deep Reinforcement Learning in Algorithmic Trading](https://mlsanity.com/paper/2509.09176/) (2025)
- [Diagonal Adaptive Non-local Observables on Quantum Neural Networks](https://mlsanity.com/paper/2605.15410/) (2026)

## Abstract (arXiv)

The development of quantum machine learning (QML) has received a lot of interest recently thanks to developments in both quantum computing (QC) and machine learning (ML). One of the ML paradigms that can be utilized to address challenging sequential decision-making issues is reinforcement learning (RL). It has been demonstrated that classical RL can successfully complete many difficult tasks. A leading method of building quantum RL agents relies on the variational quantum circuits (VQC). However, training QRL algorithms with VQCs requires significant amount of computational resources. This issue hurdles the exploration of various QRL applications. In this paper, we approach this challenge through asynchronous training QRL agents. Specifically, we choose the asynchronous training of advantage actor-critic variational quantum policies. We demonstrate the results via numerical simulations that within the tasks considered, the asynchronous training of QRL agents can reach performance comparable to or superior than classical agents with similar model sizes and architectures.