# Reinforcement Learning for Variational Quantum Circuits Design

arXiv: https://arxiv.org/abs/2409.05475 · HTML: https://mlsanity.com/paper/2409.05475/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Placed in Quantum reinforcement learning, one of 15 directions in Deep reinforcement learning methods.
- Builds on 22 of its 69 references that are in the map.
- First posted on arXiv 9 Sep 2024.

## Placed in

- Deep reinforcement learning methods / [Quantum reinforcement learning](https://mlsanity.com/deeprl/quantum-reinforcement-learning/)

## Builds on (most cited first, 22 of 22)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Variational Quantum Algorithms](https://mlsanity.com/paper/2012.09265/) (2020)
- [Barren plateaus in quantum neural network training landscapes](https://mlsanity.com/paper/1803.11173/) (2018)
- [Ising formulations of many NP problems](https://mlsanity.com/paper/1302.5843/) (2013)
- [Connecting ansatz expressibility to gradient magnitudes and barren plateaus](https://mlsanity.com/paper/2101.02138/) (2021)
- [Quantum circuit architecture search for variational quantum algorithms](https://mlsanity.com/paper/2010.10217/) (2020)
- [Effect of barren plateaus on gradient-free optimization](https://mlsanity.com/paper/2011.12245/) (2020)
- [Reinforcement learning for optimization of variational quantum circuit architectures](https://mlsanity.com/paper/2103.16089/) (2021)
- [Exploiting symmetry in variational quantum machine learning](https://mlsanity.com/paper/2205.06217/) (2022)
- [Quantum Architecture Search via Deep Reinforcement Learning](https://mlsanity.com/paper/2104.07715/) (2021)
- [Equivalence of quantum barren plateaus to cost concentration and narrow gorges](https://mlsanity.com/paper/2104.05868/) (2021)
- [Learning to Optimize Variational Quantum Circuits to Solve Combinatorial Problems](https://mlsanity.com/paper/1911.11071/) (2019)
- [Topological Quantum Compiling with Reinforcement Learning](https://mlsanity.com/paper/2004.04743/) (2020)
- [A Domain-agnostic, Noise-resistant, Hardware-efficient Evolutionary Variational Quantum Eigensolver](https://mlsanity.com/paper/1910.09694/) (2019)
- [A semi-agnostic ansatz with variable structure for quantum machine learning](https://mlsanity.com/paper/2103.06712/) (2021)
- [Policy Gradient based Quantum Approximate Optimization Algorithm](https://mlsanity.com/paper/2002.01068/) (2020)
- [Quantum Circuit Design Search](https://mlsanity.com/paper/2012.04046/) (2020)
- [Symmetry-invariant quantum machine learning force fields](https://mlsanity.com/paper/2311.11362/) (2023)
- [Performance of QAOA on Typical Instances of Constraint Satisfaction Problems with Bounded Degree](https://mlsanity.com/paper/1601.01744/) (2016)
- [Reinforcement Learning Generation of 4-Qubits Entangled States](https://mlsanity.com/paper/2204.12351/) (2022)
- [Review of Ansatz Designing Techniques for Variational Quantum Algorithms](https://mlsanity.com/paper/2212.04913/) (2022)
- [Benchmarking Adaptative Variational Quantum Algorithms on QUBO Instances](https://mlsanity.com/paper/2308.01789/) (2023)

## Built on by (most cited first, 9 of 9)

- [Learning to Measure Quantum Neural Networks](https://mlsanity.com/paper/2501.05663/) (2025)
- [Reinforcement learning with learned gadgets to tackle hard quantum problems on real hardware](https://mlsanity.com/paper/2411.00230/) (2024)
- [Quantum Long Short-term Memory with Differentiable Architecture Search](https://mlsanity.com/paper/2508.14955/) (2025)
- [Introduction to Quantum Machine Learning and Quantum Architecture Search](https://mlsanity.com/paper/2504.16131/) (2025)
- [BenchRL-QAS: Benchmarking reinforcement learning algorithms for quantum architecture search](https://mlsanity.com/paper/2507.12189/) (2025)
- [Neural Architecture Search for Quantum Autoencoders](https://mlsanity.com/paper/2511.19246/) (2025)
- [QuantumQA: Enhancing Scientific Reasoning via Physics-Consistent Dataset and Verification-Aware Reinforcement Learning](https://mlsanity.com/paper/2604.18176/) (2026)
- [Modeling quantum neural network gradient with reinforcement learning](https://mlsanity.com/paper/2609.31066/) (2026)
- [Beyond QAOA: A Review of AI and Quantum Computing for Adaptive Combinatorial Optimization](https://mlsanity.com/paper/2610.11759/) (2026)

## Abstract (arXiv)

Variational Quantum Algorithms have emerged as promising tools for solving optimization problems on quantum computers. These algorithms leverage a parametric quantum circuit called ansatz, where its parameters are adjusted by a classical optimizer with the goal of optimizing a certain cost function. However, a significant challenge lies in designing effective circuits for addressing specific problems. In this study, we leverage the powerful and flexible Reinforcement Learning paradigm to train an agent capable of autonomously generating quantum circuits that can be used as ansatzes in variational algorithms to solve optimization problems. The agent is trained on diverse problem instances, including Maximum Cut, Maximum Clique and Minimum Vertex Cover, built from different graph topologies and sizes. Our analysis of the circuits generated by the agent and the corresponding solutions shows that the proposed method is able to generate effective ansatzes. While our goal is not to propose any new specific ansatz, we observe how the agent has discovered a novel family of ansatzes effective for Maximum Cut problems, which we call Ryz-connected. We study the characteristics of one of these ansatzes by comparing it against state-of-the-art quantum algorithms across instances of varying graph topologies, sizes, and problem types. Our results indicate that the Ryz-connected circuit achieves high approximation ratios for Maximum Cut problems, further validating our proposed agent. In conclusion, our study highlights the potential of Reinforcement Learning techniques in assisting researchers to design effective quantum circuits which could have applications in a wide number of tasks.