# Differentiable Quantum Architecture Search in Asynchronous Quantum Reinforcement Learning

arXiv: https://arxiv.org/abs/2407.18202 · HTML: https://mlsanity.com/paper/2407.18202/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Placed in Quantum reinforcement learning, one of 15 directions in Deep reinforcement learning methods.
- Builds on 37 of its 68 references that are in the map.
- First posted on arXiv 25 Jul 2024.

## Placed in

- Deep reinforcement learning methods / [Quantum reinforcement learning](https://mlsanity.com/deeprl/quantum-reinforcement-learning/)

## Builds on (most cited first, 30 of 37)

- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [Variational Quantum Algorithms](https://mlsanity.com/paper/2012.09265/) (2020)
- [Variational Quantum Circuits for Deep Reinforcement Learning](https://mlsanity.com/paper/1907.00397/) (2019)
- [Quantum Long Short-Term Memory](https://mlsanity.com/paper/2009.01783/) (2020)
- [Parametrized quantum policies for reinforcement learning](https://mlsanity.com/paper/2103.05577/) (2021)
- [Quantum circuit architecture search for variational quantum algorithms](https://mlsanity.com/paper/2010.10217/) (2020)
- [Recurrent Quantum Neural Networks](https://mlsanity.com/paper/2006.14619/) (2020)
- [Reinforcement Learning with Quantum Variational Circuits](https://mlsanity.com/paper/2008.07524/) (2020)
- [Reinforcement learning for optimization of variational quantum circuit architectures](https://mlsanity.com/paper/2103.16089/) (2021)
- [Quantum Convolutional Neural Networks for High Energy Physics Data Analysis](https://mlsanity.com/paper/2012.12177/) (2020)
- [The Dawn of Quantum Natural Language Processing](https://mlsanity.com/paper/2110.06510/) (2021)
- [Quantum Architecture Search via Deep Reinforcement Learning](https://mlsanity.com/paper/2104.07715/) (2021)
- [Decentralizing Feature Extraction with Quantum Convolutional Neural Network for Automatic Speech Recognition](https://mlsanity.com/paper/2010.13309/) (2020)
- [An end-to-end trainable hybrid classical-quantum classifier](https://mlsanity.com/paper/2102.02416/) (2021)
- [Variational Quantum Reinforcement Learning via Evolutionary Optimization](https://mlsanity.com/paper/2109.00540/) (2021)
- [When BERT Meets Quantum Temporal Convolution Learning for Text Classification in Heterogeneous Computing](https://mlsanity.com/paper/2203.03550/) (2022)
- [QTN-VQC: An End-to-End Learning framework for Quantum Neural Networks](https://mlsanity.com/paper/2110.03861/) (2021)
- [PQLM -- Multilingual Decentralized Portable Quantum Language Model for Privacy Protection](https://mlsanity.com/paper/2210.03221/) (2022)
- [Quantum deep recurrent reinforcement learning](https://mlsanity.com/paper/2210.14876/) (2022)
- [Learning to Program Variational Quantum Circuits with Fast Weights](https://mlsanity.com/paper/2402.17760/) (2024)
- [Applying QNLP to sentiment analysis in finance](https://mlsanity.com/paper/2307.11788/) (2023)
- [Quantum Architecture Search via Continual Reinforcement Learning](https://mlsanity.com/paper/2112.05779/) (2021)
- [Asynchronous training of quantum reinforcement learning](https://mlsanity.com/paper/2301.05096/) (2023)
- [Quantum Neural Architecture Search with Quantum Circuits Metric and Bayesian Optimization](https://mlsanity.com/paper/2206.14115/) (2022)
- [Automated Quantum Circuit Design with Nested Monte Carlo Tree Search](https://mlsanity.com/paper/2207.00132/) (2022)
- [Variational Quantum Soft Actor-Critic](https://mlsanity.com/paper/2112.11921/) (2021)
- [Reservoir Computing via Quantum Recurrent Neural Networks](https://mlsanity.com/paper/2211.02612/) (2022)
- [Efficient quantum recurrent reinforcement learning via quantum reservoir computing](https://mlsanity.com/paper/2309.07339/) (2023)
- [Differentiable Quantum Architecture Search for Quantum Reinforcement Learning](https://mlsanity.com/paper/2309.10392/) (2023)

## Built on by (most cited first, 14 of 14)

- [Learning to Measure Quantum Neural Networks](https://mlsanity.com/paper/2501.05663/) (2025)
- [Quantum-Train-Based Distributed Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2412.08845/) (2024)
- [An Introduction to Quantum Reinforcement Learning (QRL)](https://mlsanity.com/paper/2409.05846/) (2024)
- [Differentiable Quantum Architecture Search in Quantum-Enhanced Neural Network Parameter Generation](https://mlsanity.com/paper/2505.09653/) (2025)
- [Learning to Program Quantum Measurements for Machine Learning](https://mlsanity.com/paper/2505.13525/) (2025)
- [Quantum Long Short-term Memory with Differentiable Architecture Search](https://mlsanity.com/paper/2508.14955/) (2025)
- [Introduction to Quantum Machine Learning and Quantum Architecture Search](https://mlsanity.com/paper/2504.16131/) (2025)
- [Quantum Machine Learning: An Interplay Between Quantum Computing and Machine Learning](https://mlsanity.com/paper/2411.09403/) (2024)
- [Recursive QLSTM with Dynamic Variational Quantum Circuit Adaptation](https://mlsanity.com/paper/2606.24932/) (2026)
- [Neural Architecture Search for Quantum Autoencoders](https://mlsanity.com/paper/2511.19246/) (2025)
- [Differentiable Architecture Search for Adversarially Robust Quantum Computer Vision](https://mlsanity.com/paper/2601.18058/) (2026)
- [QCS-ADME: Quantum Circuit Search for Drug Property Prediction with Imbalanced Data and Regression Adaptation](https://mlsanity.com/paper/2503.01927/) (2025)
- [Special-Unitary Parameterization for Trainable Variational Quantum Circuits](https://mlsanity.com/paper/2507.05535/) (2025)
- [Diversity-Aware Ensemble of Quantum-Classical Models via Combinatorial Fusion](https://mlsanity.com/paper/2610.13808/) (2026)

## Abstract (arXiv)

The emergence of quantum reinforcement learning (QRL) is propelled by advancements in quantum computing (QC) and machine learning (ML), particularly through quantum neural networks (QNN) built on variational quantum circuits (VQC). These advancements have proven successful in addressing sequential decision-making tasks. However, constructing effective QRL models demands significant expertise due to challenges in designing quantum circuit architectures, including data encoding and parameterized circuits, which profoundly influence model performance. In this paper, we propose addressing this challenge with differentiable quantum architecture search (DiffQAS), enabling trainable circuit parameters and structure weights using gradient-based optimization. Furthermore, we enhance training efficiency through asynchronous reinforcement learning (RL) methods facilitating parallel training. Through numerical simulations, we demonstrate that our proposed DiffQAS-QRL approach achieves performance comparable to manually-crafted circuit architectures across considered environments, showcasing stability across diverse scenarios. This methodology offers a pathway for designing QRL models without extensive quantum knowledge, ensuring robust performance and fostering broader application of QRL.