# Variational Quantum Reinforcement Learning via Evolutionary Optimization

arXiv: https://arxiv.org/abs/2109.00540 · HTML: https://mlsanity.com/paper/2109.00540/

## Key facts

- Cited by 50 papers in the map (data through 14 Oct 2026).
- Builds on 47 of its 125 references that are in the map.
- First posted on arXiv 1 Sep 2021.

## Builds on (most cited first, 30 of 47)

- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model](https://mlsanity.com/paper/1911.08265/) (2019)
- [Variational Quantum Algorithms](https://mlsanity.com/paper/2012.09265/) (2020)
- [Supervised learning with quantum enhanced feature spaces](https://mlsanity.com/paper/1804.11326/) (2018)
- [Parameterized quantum circuits as machine learning models](https://mlsanity.com/paper/1906.07682/) (2019)
- [Agent57: Outperforming the Atari Human Benchmark](https://mlsanity.com/paper/2003.13350/) (2020)
- [Variational Quantum Circuits for Deep Reinforcement Learning](https://mlsanity.com/paper/1907.00397/) (2019)
- [Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learning](https://mlsanity.com/paper/1712.06567/) (2017)
- [Quantum generative adversarial networks](https://mlsanity.com/paper/1804.08641/) (2018)
- [Quantum Long Short-Term Memory](https://mlsanity.com/paper/2009.01783/) (2020)
- [Transfer learning in hybrid classical-quantum neural networks](https://mlsanity.com/paper/1912.08278/) (2019)
- [Improving Exploration in Evolution Strategies for Deep Reinforcement Learning via a Population of Novelty-Seeking Agents](https://mlsanity.com/paper/1712.06560/) (2017)
- [Unsupervised Generative Modeling Using Matrix Product States](https://mlsanity.com/paper/1709.01662/) (2017)
- [Federated Quantum Machine Learning](https://mlsanity.com/paper/2103.12010/) (2021)
- [Recurrent Quantum Neural Networks](https://mlsanity.com/paper/2006.14619/) (2020)
- [Noisy intermediate-scale quantum (NISQ) algorithms](https://mlsanity.com/paper/2101.08448/) (2021)
- [Reinforcement Learning with Quantum Variational Circuits](https://mlsanity.com/paper/2008.07524/) (2020)
- [Towards Quantum Machine Learning with Tensor Networks](https://mlsanity.com/paper/1803.11537/) (2018)
- [Tree Tensor Networks for Generative Modeling](https://mlsanity.com/paper/1901.02217/) (2019)
- [Quantum Convolutional Neural Networks for High Energy Physics Data Analysis](https://mlsanity.com/paper/2012.12177/) (2020)
- [Quantum Architecture Search via Deep Reinforcement Learning](https://mlsanity.com/paper/2104.07715/) (2021)
- [Decentralizing Feature Extraction with Quantum Convolutional Neural Network for Automatic Speech Recognition](https://mlsanity.com/paper/2010.13309/) (2020)
- [Quantum Federated Learning with Quantum Data](https://mlsanity.com/paper/2106.00005/) (2021)
- [Learning Relevant Features of Data with Multi-scale Tensor Networks](https://mlsanity.com/paper/1801.00315/) (2017)
- [An end-to-end trainable hybrid classical-quantum classifier](https://mlsanity.com/paper/2102.02416/) (2021)
- [Active learning machine learns to create new quantum experiments](https://mlsanity.com/paper/1706.00868/) (2017)
- [Machine Learning by Unitary Tensor Network of Hierarchical Tree Structure](https://mlsanity.com/paper/1710.04833/) (2017)
- [From probabilistic graphical models to generalized tensor networks for supervised learning](https://mlsanity.com/paper/1806.05964/) (2018)
- [Optimizing Quantum Error Correction Codes with Reinforcement Learning](https://mlsanity.com/paper/1812.08451/) (2018)
- [TensorNetwork for Machine Learning](https://mlsanity.com/paper/1906.06329/) (2019)

## Built on by (most cited first, 30 of 50)

- [A Survey on Quantum Reinforcement Learning](https://mlsanity.com/paper/2211.03464/) (2022)
- [Quantum deep recurrent reinforcement learning](https://mlsanity.com/paper/2210.14876/) (2022)
- [Learning to Program Variational Quantum Circuits with Fast Weights](https://mlsanity.com/paper/2402.17760/) (2024)
- [Quantum Architecture Search via Continual Reinforcement Learning](https://mlsanity.com/paper/2112.05779/) (2021)
- [Asynchronous training of quantum reinforcement learning](https://mlsanity.com/paper/2301.05096/) (2023)
- [Tensor networks for quantum machine learning](https://mlsanity.com/paper/2303.11735/) (2023)
- [Quantum Deep Reinforcement Learning for Robot Navigation Tasks](https://mlsanity.com/paper/2202.12180/) (2022)
- [Quantum policy gradient algorithms](https://mlsanity.com/paper/2212.09328/) (2022)
- [Federated Quantum Long Short-term Memory (FedQLSTM)](https://mlsanity.com/paper/2312.14309/) (2023)
- [Learning to Measure Quantum Neural Networks](https://mlsanity.com/paper/2501.05663/) (2025)
- [Reservoir Computing via Quantum Recurrent Neural Networks](https://mlsanity.com/paper/2211.02612/) (2022)
- [Efficient quantum recurrent reinforcement learning via quantum reservoir computing](https://mlsanity.com/paper/2309.07339/) (2023)
- [Differentiable Quantum Architecture Search in Asynchronous Quantum Reinforcement Learning](https://mlsanity.com/paper/2407.18202/) (2024)
- [Quantum-Train-Based Distributed Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2412.08845/) (2024)
- [Differentiable Quantum Architecture Search for Quantum Reinforcement Learning](https://mlsanity.com/paper/2309.10392/) (2023)
- [Quantum Machine Learning Architecture Search via Deep Reinforcement Learning](https://mlsanity.com/paper/2407.20147/) (2024)
- [A Primer on Quantum Machine Learning](https://mlsanity.com/paper/2511.15969/) (2025)
- [QClusformer: A Quantum Transformer-based Framework for Unsupervised Visual Clustering](https://mlsanity.com/paper/2405.19722/) (2024)
- [Diffusion-Inspired Quantum Noise Mitigation in Parameterized Quantum Circuits](https://mlsanity.com/paper/2406.00843/) (2024)
- [Quantum Advantage Actor-Critic for Reinforcement Learning](https://mlsanity.com/paper/2401.07043/) (2024)
- [Quantum Gradient Class Activation Map for Model Interpretability](https://mlsanity.com/paper/2408.05899/) (2024)
- [Evolutionary Optimization for Designing Variational Quantum Circuits with High Model Capacity](https://mlsanity.com/paper/2412.12484/) (2024)
- [A Study on Optimization Techniques for Variational Quantum Circuits in Reinforcement Learning](https://mlsanity.com/paper/2405.12354/) (2024)
- [Hierarchical Quantum Control Gates for Functional MRI Understanding](https://mlsanity.com/paper/2408.03596/) (2024)
- [Expressibility-induced Concentration of Quantum Neural Tangent Kernels](https://mlsanity.com/paper/2311.04965/) (2023)
- [An Introduction to Quantum Reinforcement Learning (QRL)](https://mlsanity.com/paper/2409.05846/) (2024)
- [Adaptive Non-local Observable on Quantum Neural Networks](https://mlsanity.com/paper/2504.13414/) (2025)
- [Differentiable Quantum Architecture Search in Quantum-Enhanced Neural Network Parameter Generation](https://mlsanity.com/paper/2505.09653/) (2025)
- [Learning to Program Quantum Measurements for Machine Learning](https://mlsanity.com/paper/2505.13525/) (2025)
- [QMoE: A Quantum Mixture of Experts Framework for Scalable Quantum Neural Networks](https://mlsanity.com/paper/2507.05190/) (2025)

## Abstract (arXiv)

Recent advance in classical reinforcement learning (RL) and quantum computation (QC) points to a promising direction of performing RL on a quantum computer. However, potential applications in quantum RL are limited by the number of qubits available in the modern quantum devices. Here we present two frameworks of deep quantum RL tasks using a gradient-free evolution optimization: First, we apply the amplitude encoding scheme to the Cart-Pole problem; Second, we propose a hybrid framework where the quantum RL agents are equipped with hybrid tensor network-variational quantum circuit (TN-VQC) architecture to handle inputs with dimensions exceeding the number of qubits. This allows us to perform quantum RL on the MiniGrid environment with 147-dimensional inputs. We demonstrate the quantum advantage of parameter saving using the amplitude encoding. The hybrid TN-VQC architecture provides a natural way to perform efficient compression of the input dimension, enabling further quantum RL applications on noisy intermediate-scale quantum devices.