# Quantum Architecture Search via Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/2104.07715 · HTML: https://mlsanity.com/paper/2104.07715/

## Key facts

- Cited by 60 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 94 references that are in the map.
- First posted on arXiv 15 Apr 2021.

## Builds on (most cited first, 30 of 37)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [Neural Architecture Search with Reinforcement Learning](https://mlsanity.com/paper/1611.01578/) (2016)
- [Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model](https://mlsanity.com/paper/1911.08265/) (2019)
- [Learning Transferable Architectures for Scalable Image Recognition](https://mlsanity.com/paper/1707.07012/) (2017)
- [An overview of gradient descent optimization algorithms](https://mlsanity.com/paper/1609.04747/) (2016)
- [Efficient Neural Architecture Search via Parameter Sharing](https://mlsanity.com/paper/1802.03268/) (2018)
- [Supervised learning with quantum enhanced feature spaces](https://mlsanity.com/paper/1804.11326/) (2018)
- [Parameterized quantum circuits as machine learning models](https://mlsanity.com/paper/1906.07682/) (2019)
- [Agent57: Outperforming the Atari Human Benchmark](https://mlsanity.com/paper/2003.13350/) (2020)
- [Designing Neural Network Architectures using Reinforcement Learning](https://mlsanity.com/paper/1611.02167/) (2016)
- [Variational Quantum Circuits for Deep Reinforcement Learning](https://mlsanity.com/paper/1907.00397/) (2019)
- [Quantum generative adversarial networks](https://mlsanity.com/paper/1804.08641/) (2018)
- [Quantum Long Short-Term Memory](https://mlsanity.com/paper/2009.01783/) (2020)
- [Transfer learning in hybrid classical-quantum neural networks](https://mlsanity.com/paper/1912.08278/) (2019)
- [Federated Quantum Machine Learning](https://mlsanity.com/paper/2103.12010/) (2021)
- [Recurrent Quantum Neural Networks](https://mlsanity.com/paper/2006.14619/) (2020)
- [Reinforcement Learning with Quantum Variational Circuits](https://mlsanity.com/paper/2008.07524/) (2020)
- [Reinforcement learning for optimization of variational quantum circuit architectures](https://mlsanity.com/paper/2103.16089/) (2021)
- [Quantum Convolutional Neural Networks for High Energy Physics Data Analysis](https://mlsanity.com/paper/2012.12177/) (2020)
- [Decentralizing Feature Extraction with Quantum Convolutional Neural Network for Automatic Speech Recognition](https://mlsanity.com/paper/2010.13309/) (2020)
- [Learning to learn with quantum neural networks via classical neural networks](https://mlsanity.com/paper/1907.05415/) (2019)
- [Universal Quantum Control through Deep Reinforcement Learning](https://mlsanity.com/paper/1803.01857/) (2018)
- [Path-Level Network Transformation for Efficient Architecture Search](https://mlsanity.com/paper/1806.02639/) (2018)
- [Optimizing Quantum Error Correction Codes with Reinforcement Learning](https://mlsanity.com/paper/1812.08451/) (2018)
- [Hybrid quantum-classical classifier based on tensor network and variational quantum circuit](https://mlsanity.com/paper/2011.14651/) (2020)
- [Classification with Quantum Machine Learning: A Survey](https://mlsanity.com/paper/2006.12270/) (2020)
- [Hybrid Quantum-Classical Graph Convolutional Network](https://mlsanity.com/paper/2101.06189/) (2021)

## Built on by (most cited first, 30 of 60)

- [Variational Quantum Reinforcement Learning via Evolutionary Optimization](https://mlsanity.com/paper/2109.00540/) (2021)
- [Curriculum reinforcement learning for quantum architecture search under hardware errors](https://mlsanity.com/paper/2402.03500/) (2024)
- [KANQAS: Kolmogorov-Arnold Network for Quantum Architecture Search](https://mlsanity.com/paper/2406.17630/) (2024)
- [Quantum Circuit Optimization with AlphaTensor](https://mlsanity.com/paper/2402.14396/) (2024)
- [Quantum Architecture Search: A Survey](https://mlsanity.com/paper/2406.06210/) (2024)
- [Quantum Architecture Search via Continual Reinforcement Learning](https://mlsanity.com/paper/2112.05779/) (2021)
- [Quantum Neural Architecture Search with Quantum Circuits Metric and Bayesian Optimization](https://mlsanity.com/paper/2206.14115/) (2022)
- [Automated Quantum Circuit Design with Nested Monte Carlo Tree Search](https://mlsanity.com/paper/2207.00132/) (2022)
- [Learning to Measure Quantum Neural Networks](https://mlsanity.com/paper/2501.05663/) (2025)
- [Differentiable Quantum Architecture Search in Asynchronous Quantum Reinforcement Learning](https://mlsanity.com/paper/2407.18202/) (2024)
- [Quantum Machine Learning Architecture Search via Deep Reinforcement Learning](https://mlsanity.com/paper/2407.20147/) (2024)
- [Enhancing variational quantum state diagonalization using reinforcement learning techniques](https://mlsanity.com/paper/2306.11086/) (2023)
- [Reinforcement Learning for Generative AI: A Survey](https://mlsanity.com/paper/2308.14328/) (2023)
- [Monte Carlo Tree Search based Hybrid Optimization of Variational Quantum Circuits](https://mlsanity.com/paper/2203.16707/) (2022)
- [Reinforcement learning-assisted quantum architecture search for variational quantum algorithms](https://mlsanity.com/paper/2402.13754/) (2024)
- [Challenges for Reinforcement Learning in Quantum Circuit Design](https://mlsanity.com/paper/2312.11337/) (2023)
- [Reinforcement learning-based architecture search for quantum machine learning](https://mlsanity.com/paper/2406.02717/) (2024)
- [Reinforcement Learning for Variational Quantum Circuits Design](https://mlsanity.com/paper/2409.05475/) (2024)
- [Evolutionary Optimization for Designing Variational Quantum Circuits with High Model Capacity](https://mlsanity.com/paper/2412.12484/) (2024)
- [Quantum Circuit Synthesis and Compilation Optimization: Overview and Prospects](https://mlsanity.com/paper/2407.00736/) (2024)
- [TensorRL-QAS: Reinforcement learning with tensor networks for improved quantum architecture search](https://mlsanity.com/paper/2505.09371/) (2025)
- [Differentiable Quantum Architecture Search in Quantum-Enhanced Neural Network Parameter Generation](https://mlsanity.com/paper/2505.09653/) (2025)
- [Efficient and practical quantum compiler towards multi-qubit systems with deep reinforcement learning](https://mlsanity.com/paper/2204.06904/) (2022)
- [Power of Quantum Generative Learning](https://mlsanity.com/paper/2205.04730/) (2022)
- [The quantum cost function concentration dependency on the parametrization expressivity](https://mlsanity.com/paper/2301.06883/) (2023)
- [Quantum Circuit Design using a Progressive Widening Enhanced Monte Carlo Tree Search](https://mlsanity.com/paper/2502.03962/) (2025)
- [Quantum Reinforcement Learning Trading Agent for Sector Rotation in the Taiwan Stock Market](https://mlsanity.com/paper/2506.20930/) (2025)
- [Quantum Architecture Search with Unsupervised Representation Learning](https://mlsanity.com/paper/2401.11576/) (2024)
- [MG-Net: Learn to Customize QAOA with Circuit Depth Awareness](https://mlsanity.com/paper/2409.18692/) (2024)
- [Q-Fusion: Diffusing Quantum Circuits](https://mlsanity.com/paper/2504.20794/) (2025)

## Abstract (arXiv)

Recent advances in quantum computing have drawn considerable attention to building realistic application for and using quantum computers. However, designing a suitable quantum circuit architecture requires expert knowledge. For example, it is non-trivial to design a quantum gate sequence for generating a particular quantum state with as fewer gates as possible. We propose a quantum architecture search framework with the power of deep reinforcement learning (DRL) to address this challenge. In the proposed framework, the DRL agent can only access the Pauli-X, Y, Z expectation values and a predefined set of quantum operations for learning the target quantum state, and is optimized by the advantage actor-critic (A2C) and proximal policy optimization (PPO) algorithms. We demonstrate a successful generation of quantum gate sequences for multi-qubit GHZ states without encoding any knowledge of quantum physics in the agent. The design of our framework is rather general and can be employed with other DRL architectures or optimization methods to study gate synthesis and compilation for many quantum states.