# Reinforcement learning for optimization of variational quantum circuit architectures

arXiv: https://arxiv.org/abs/2103.16089 · HTML: https://mlsanity.com/paper/2103.16089/

## Key facts

- Cited by 77 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 61 references that are in the map.
- First posted on arXiv 30 Mar 2021.

## Builds on (most cited first, 16 of 16)

- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Variational Quantum Algorithms](https://mlsanity.com/paper/2012.09265/) (2020)
- [Parameterized quantum circuits as machine learning models](https://mlsanity.com/paper/1906.07682/) (2019)
- [First return, then explore](https://mlsanity.com/paper/2004.12919/) (2020)
- [Automatic Curriculum Learning For Deep RL: A Short Survey](https://mlsanity.com/paper/2003.04664/) (2020)
- [Quantum generative adversarial networks](https://mlsanity.com/paper/1804.08641/) (2018)
- [A survey on intrinsic motivation in reinforcement learning](https://mlsanity.com/paper/1908.06976/) (2019)
- [Learning to Optimize Variational Quantum Circuits to Solve Combinatorial Problems](https://mlsanity.com/paper/1911.11071/) (2019)
- [Enabling Multi-programming Mechanism for Quantum Computing in the NISQ Era](https://mlsanity.com/paper/2102.05321/) (2021)
- [A Domain-agnostic, Noise-resistant, Hardware-efficient Evolutionary Variational Quantum Eigensolver](https://mlsanity.com/paper/1910.09694/) (2019)
- [Quantum Circuit Design Search](https://mlsanity.com/paper/2012.04046/) (2020)
- [Interferobot: aligning an optical interferometer by a reinforcement learning agent](https://mlsanity.com/paper/2006.02252/) (2020)
- [Quantum Logic Gate Synthesis as a Markov Decision Process](https://mlsanity.com/paper/1912.12002/) (2019)
- [Approximation of quantum control correction scheme using deep neural networks](https://mlsanity.com/paper/1803.05193/) (2018)
- [Exploring the Quantum Speed Limit with Computer Games](https://mlsanity.com/paper/1506.09091/) (2015)
- [Geometrical versus time-series representation of data in quantum control learning](https://mlsanity.com/paper/1803.05169/) (2018)

## Built on by (most cited first, 30 of 77)

- [Quantum Architecture Search via Deep Reinforcement Learning](https://mlsanity.com/paper/2104.07715/) (2021)
- [Curriculum reinforcement learning for quantum architecture search under hardware errors](https://mlsanity.com/paper/2402.03500/) (2024)
- [KANQAS: Kolmogorov-Arnold Network for Quantum Architecture Search](https://mlsanity.com/paper/2406.17630/) (2024)
- [Quantum Circuit Optimization with AlphaTensor](https://mlsanity.com/paper/2402.14396/) (2024)
- [Quantum circuit synthesis with diffusion models](https://mlsanity.com/paper/2311.02041/) (2023)
- [Quantum Architecture Search: A Survey](https://mlsanity.com/paper/2406.06210/) (2024)
- [QTN-VQC: An End-to-End Learning framework for Quantum Neural Networks](https://mlsanity.com/paper/2110.03861/) (2021)
- [Theoretical Error Performance Analysis for Variational Quantum Circuit Based Functional Regression](https://mlsanity.com/paper/2206.04804/) (2022)
- [Quantum Architecture Search via Continual Reinforcement Learning](https://mlsanity.com/paper/2112.05779/) (2021)
- [Concentration of Data Encoding in Parameterized Quantum Circuits](https://mlsanity.com/paper/2206.08273/) (2022)
- [Differentiable Quantum Architecture Search in Asynchronous Quantum Reinforcement Learning](https://mlsanity.com/paper/2407.18202/) (2024)
- [Quantum Machine Learning Architecture Search via Deep Reinforcement Learning](https://mlsanity.com/paper/2407.20147/) (2024)
- [Meta-Designing Quantum Experiments with Language Models](https://mlsanity.com/paper/2406.02470/) (2024)
- [Enhancing variational quantum state diagonalization using reinforcement learning techniques](https://mlsanity.com/paper/2306.11086/) (2023)
- [Reinforcement Learning for Generative AI: A Survey](https://mlsanity.com/paper/2308.14328/) (2023)
- [Quantum Mixed-State Self-Attention Network](https://mlsanity.com/paper/2403.02871/) (2024)
- [Automatic and effective discovery of quantum kernels](https://mlsanity.com/paper/2209.11144/) (2022)
- [Quarl: A Learning-Based Quantum Circuit Optimizer](https://mlsanity.com/paper/2307.10120/) (2023)
- [A Reinforcement Learning Environment for Directed Quantum Circuit Synthesis](https://mlsanity.com/paper/2401.07054/) (2024)
- [Reinforcement learning-assisted quantum architecture search for variational quantum algorithms](https://mlsanity.com/paper/2402.13754/) (2024)
- [Quantum Visual Feature Encoding Revisited](https://mlsanity.com/paper/2405.19725/) (2024)
- [Challenges for Reinforcement Learning in Quantum Circuit Design](https://mlsanity.com/paper/2312.11337/) (2023)
- [Quantum Generative Diffusion Model: A Fully Quantum-Mechanical Model for Generating Quantum State Ensemble](https://mlsanity.com/paper/2401.07039/) (2024)
- [Reinforcement learning-based architecture search for quantum machine learning](https://mlsanity.com/paper/2406.02717/) (2024)
- [Reinforcement Learning for Variational Quantum Circuits Design](https://mlsanity.com/paper/2409.05475/) (2024)
- [Evolutionary Optimization for Designing Variational Quantum Circuits with High Model Capacity](https://mlsanity.com/paper/2412.12484/) (2024)
- [Quantum Circuit Synthesis and Compilation Optimization: Overview and Prospects](https://mlsanity.com/paper/2407.00736/) (2024)
- [Improving Quantum Circuit Synthesis with Machine Learning](https://mlsanity.com/paper/2306.05622/) (2023)
- [Neural auto-designer for enhanced quantum kernels](https://mlsanity.com/paper/2401.11098/) (2024)
- [Physics-Informed Bayesian Optimization of Variational Quantum Circuits](https://mlsanity.com/paper/2406.06150/) (2024)

## Abstract (arXiv)

The study of Variational Quantum Eigensolvers (VQEs) has been in the spotlight in recent times as they may lead to real-world applications of near-term quantum devices. However, their performance depends on the structure of the used variational ansatz, which requires balancing the depth and expressivity of the corresponding circuit. In recent years, various methods for VQE structure optimization have been introduced but the capacities of machine learning to aid with this problem has not yet been fully investigated. In this work, we propose a reinforcement learning algorithm that autonomously explores the space of possible ans"atze, identifying economic circuits which still yield accurate ground energy estimates. The algorithm is intrinsically motivated, and it incrementally improves the accuracy of the result while minimizing the circuit depth. We showcase the performance of our algorithm on the problem of estimating the ground-state energy of lithium hydride (LiH). In this well-known benchmark problem, we achieve chemical accuracy, as well as state-of-the-art results in terms of circuit depth.