# Differentiable Quantum Architecture Search for Quantum Reinforcement Learning

arXiv: https://arxiv.org/abs/2309.10392 · HTML: https://mlsanity.com/paper/2309.10392/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 25 references that are in the map.
- First posted on arXiv 19 Sep 2023.

## Builds on (most cited first, 10 of 10)

- [Variational Quantum Algorithms](https://mlsanity.com/paper/2012.09265/) (2020)
- [Supervised learning with quantum enhanced feature spaces](https://mlsanity.com/paper/1804.11326/) (2018)
- [Variational Quantum Circuits for Deep Reinforcement Learning](https://mlsanity.com/paper/1907.00397/) (2019)
- [Efficient Learning for Deep Quantum Neural Networks](https://mlsanity.com/paper/1902.10445/) (2019)
- [The Born Supremacy: Quantum Advantage and Training of an Ising Born Machine](https://mlsanity.com/paper/1904.02214/) (2019)
- [Variational Quantum Reinforcement Learning via Evolutionary Optimization](https://mlsanity.com/paper/2109.00540/) (2021)
- [Quantum Wasserstein Generative Adversarial Networks](https://mlsanity.com/paper/1911.00111/) (2019)
- [Quantum-Classical Hybrid Machine Learning for Image Classification (ICCAD Special Session Paper)](https://mlsanity.com/paper/2109.02862/) (2021)
- [Variational Quantum Circuit Model for Knowledge Graphs Embedding](https://mlsanity.com/paper/1903.00556/) (2019)
- [Differentiable Architecture Search for Reinforcement Learning](https://mlsanity.com/paper/2106.02229/) (2021)

## Built on by (most cited first, 13 of 13)

- [Quantum Architecture Search: A Survey](https://mlsanity.com/paper/2406.06210/) (2024)
- [Learning to Measure Quantum Neural Networks](https://mlsanity.com/paper/2501.05663/) (2025)
- [Differentiable Quantum Architecture Search in Asynchronous Quantum Reinforcement Learning](https://mlsanity.com/paper/2407.18202/) (2024)
- [Quantum Machine Learning Architecture Search via Deep Reinforcement Learning](https://mlsanity.com/paper/2407.20147/) (2024)
- [Evolutionary Optimization for Designing Variational Quantum Circuits with High Model Capacity](https://mlsanity.com/paper/2412.12484/) (2024)
- [An Introduction to Quantum Reinforcement Learning (QRL)](https://mlsanity.com/paper/2409.05846/) (2024)
- [Differentiable Quantum Architecture Search in Quantum-Enhanced Neural Network Parameter Generation](https://mlsanity.com/paper/2505.09653/) (2025)
- [Quantum Long Short-term Memory with Differentiable Architecture Search](https://mlsanity.com/paper/2508.14955/) (2025)
- [Introduction to Quantum Machine Learning and Quantum Architecture Search](https://mlsanity.com/paper/2504.16131/) (2025)
- [Quantum Machine Learning: An Interplay Between Quantum Computing and Machine Learning](https://mlsanity.com/paper/2411.09403/) (2024)
- [Neural Architecture Search for Quantum Autoencoders](https://mlsanity.com/paper/2511.19246/) (2025)
- [Differentiable Architecture Search for Adversarially Robust Quantum Computer Vision](https://mlsanity.com/paper/2601.18058/) (2026)
- [Integrated Encoding and Quantization to Enhance Quanvolutional Neural Networks](https://mlsanity.com/paper/2410.05777/) (2024)

## Abstract (arXiv)

Differentiable quantum architecture search (DQAS) is a gradient-based framework to design quantum circuits automatically in the NISQ era. It was motivated by such as low fidelity of quantum hardware, low flexibility of circuit architecture, high circuit design cost, barren plateau (BP) problem, and periodicity of weights. People used it to address error mitigation, unitary decomposition, and quantum approximation optimization problems based on fixed datasets. Quantum reinforcement learning (QRL) is a part of quantum machine learning and often has various data. QRL usually uses a manually designed circuit. However, the pre-defined circuit needs more flexibility for different tasks, and the circuit design based on various datasets could become intractable in the case of a large circuit. The problem of whether DQAS can be applied to quantum deep Q-learning with various datasets is still open. The main target of this work is to discover the capability of DQAS to solve quantum deep Q-learning problems. We apply a gradient-based framework DQAS on reinforcement learning tasks and evaluate it in two different environments - cart pole and frozen lake. It contains input- and output weights, progressive search, and other new features. The experiments conclude that DQAS can design quantum circuits automatically and efficiently. The evaluation results show significant outperformance compared to the manually designed circuit. Furthermore, the performance of the automatically created circuit depends on whether the super-circuit learned well during the training process. This work is the first to show that gradient-based quantum architecture search is applicable to QRL tasks.