# Curriculum reinforcement learning for quantum architecture search under hardware errors

arXiv: https://arxiv.org/abs/2402.03500 · HTML: https://mlsanity.com/paper/2402.03500/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- Placed in Quantum reinforcement learning, one of 15 directions in Deep reinforcement learning methods.
- Builds on 12 of its 69 references that are in the map.
- First posted on arXiv 5 Feb 2024.

## Placed in

- Deep reinforcement learning methods / [Quantum reinforcement learning](https://mlsanity.com/deeprl/quantum-reinforcement-learning/)

## Builds on (most cited first, 12 of 12)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Variational Quantum Algorithms](https://mlsanity.com/paper/2012.09265/) (2020)
- [Noise-Induced Barren Plateaus in Variational Quantum Algorithms](https://mlsanity.com/paper/2007.14384/) (2020)
- [QuantumNAS: Noise-Adaptive Search for Robust Quantum Circuits](https://mlsanity.com/paper/2107.10845/) (2021)
- [Quantum circuit architecture search for variational quantum algorithms](https://mlsanity.com/paper/2010.10217/) (2020)
- [Reinforcement learning for optimization of variational quantum circuit architectures](https://mlsanity.com/paper/2103.16089/) (2021)
- [Quantum Architecture Search via Deep Reinforcement Learning](https://mlsanity.com/paper/2104.07715/) (2021)
- [A Domain-agnostic, Noise-resistant, Hardware-efficient Evolutionary Variational Quantum Eigensolver](https://mlsanity.com/paper/1910.09694/) (2019)
- [Stochastic noise can be helpful for variational quantum algorithms](https://mlsanity.com/paper/2210.06723/) (2022)
- [Non-trivial symmetries in quantum landscapes and their resilience to quantum noise](https://mlsanity.com/paper/2011.08763/) (2020)
- [Qubit Routing using Graph Neural Network aided Monte Carlo Tree Search](https://mlsanity.com/paper/2104.01992/) (2021)
- [Multi-objective evolutionary algorithms for quantum circuit discovery](https://mlsanity.com/paper/1812.04458/) (2018)

## Built on by (most cited first, 30 of 37)

- [KANQAS: Kolmogorov-Arnold Network for Quantum Architecture Search](https://mlsanity.com/paper/2406.17630/) (2024)
- [Quantum Architecture Search: A Survey](https://mlsanity.com/paper/2406.06210/) (2024)
- [Reinforcement learning-assisted quantum architecture search for variational quantum algorithms](https://mlsanity.com/paper/2402.13754/) (2024)
- [Quantum Circuit Synthesis and Compilation Optimization: Overview and Prospects](https://mlsanity.com/paper/2407.00736/) (2024)
- [Reinforcement learning with learned gadgets to tackle hard quantum problems on real hardware](https://mlsanity.com/paper/2411.00230/) (2024)
- [TensorRL-QAS: Reinforcement learning with tensor networks for improved quantum architecture search](https://mlsanity.com/paper/2505.09371/) (2025)
- [Differentiable Quantum Architecture Search in Quantum-Enhanced Neural Network Parameter Generation](https://mlsanity.com/paper/2505.09653/) (2025)
- [Quantum Circuit Design using a Progressive Widening Enhanced Monte Carlo Tree Search](https://mlsanity.com/paper/2502.03962/) (2025)
- [Layerwise Federated Learning for Heterogeneous Quantum Clients using Quorus](https://mlsanity.com/paper/2510.06228/) (2025)
- [Leveraging Diffusion Models for Parameterized Quantum Circuit Generation](https://mlsanity.com/paper/2505.20863/) (2025)
- [Quantum Long Short-term Memory with Differentiable Architecture Search](https://mlsanity.com/paper/2508.14955/) (2025)
- [FlowQ-Net: A Generative Framework for Automated Quantum Circuit Design](https://mlsanity.com/paper/2510.26688/) (2025)
- [Gated QKAN-FWP: Scalable Quantum-inspired Sequence Learning](https://mlsanity.com/paper/2605.06734/) (2026)
- [Generative Quantum-inspired Kolmogorov-Arnold Eigensolver](https://mlsanity.com/paper/2605.04604/) (2026)
- [Improving thermal state preparation of Sachdev-Ye-Kitaev model with reinforcement learning on quantum hardware](https://mlsanity.com/paper/2501.11454/) (2025)
- [BenchRL-QAS: Benchmarking reinforcement learning algorithms for quantum architecture search](https://mlsanity.com/paper/2507.12189/) (2025)
- [How to find expressible and trainable parameterized quantum circuits?](https://mlsanity.com/paper/2603.14451/) (2026)
- [SuperEncoder: Towards Universal Neural Approximate Quantum State Preparation](https://mlsanity.com/paper/2408.05435/) (2024)
- [Enhancing Quantum Memory Lifetime with Measurement-Free Local Error Correction and Reinforcement Learning](https://mlsanity.com/paper/2408.09524/) (2024)
- [Reinforcement learning to learn quantum states for Heisenberg scaling accuracy](https://mlsanity.com/paper/2412.02334/) (2024)
- [RhoDARTS: Differentiable Quantum Architecture Search with Density Matrix Simulations](https://mlsanity.com/paper/2506.03697/) (2025)
- [Neural Architecture Search Algorithms for Quantum Autoencoders](https://mlsanity.com/paper/2509.15451/) (2025)
- [DistributedEstimator: Distributed Training of Quantum Neural Networks via Circuit Cutting](https://mlsanity.com/paper/2602.16233/) (2026)
- [QFlowNet: Fast, Diverse, and Efficient Unitary Synthesis with Generative Flow Networks](https://mlsanity.com/paper/2603.03045/) (2026)
- [Replay-buffer engineering for noise-aware quantum circuit optimization](https://mlsanity.com/paper/2604.21863/) (2026)
- [Hybrid Classical-Quantum architecture for vectorised image classification of hand-written sketches](https://mlsanity.com/paper/2407.06416/) (2024)
- [QCS-ADME: Quantum Circuit Search for Drug Property Prediction with Imbalanced Data and Regression Adaptation](https://mlsanity.com/paper/2503.01927/) (2025)
- [Adaptive Learning Systems: Personalized Curriculum Design Using LLM-Powered Analytics](https://mlsanity.com/paper/2507.18949/) (2025)
- [Quantum Architecture Search for Solving Quantum Machine Learning Tasks](https://mlsanity.com/paper/2509.11198/) (2025)
- [Differentiable Logical Programming for Quantum Circuit Discovery and Optimization](https://mlsanity.com/paper/2602.08880/) (2026)

## Abstract (arXiv)

The key challenge in the noisy intermediate-scale quantum era is finding useful circuits compatible with current device limitations. Variational quantum algorithms (VQAs) offer a potential solution by fixing the circuit architecture and optimizing individual gate parameters in an external loop. However, parameter optimization can become intractable, and the overall performance of the algorithm depends heavily on the initially chosen circuit architecture. Several quantum architecture search (QAS) algorithms have been developed to design useful circuit architectures automatically. In the case of parameter optimization alone, noise effects have been observed to dramatically influence the performance of the optimizer and final outcomes, which is a key line of study. However, the effects of noise on the architecture search, which could be just as critical, are poorly understood. This work addresses this gap by introducing a curriculum-based reinforcement learning QAS (CRLQAS) algorithm designed to tackle challenges in realistic VQA deployment. The algorithm incorporates (i) a 3D architecture encoding and restrictions on environment dynamics to explore the search space of possible circuits efficiently, (ii) an episode halting scheme to steer the agent to find shorter circuits, and (iii) a novel variant of simultaneous perturbation stochastic approximation as an optimizer for faster convergence. To facilitate studies, we developed an optimized simulator for our algorithm, significantly improving computational efficiency in simulating noisy quantum circuits by employing the Pauli-transfer matrix formalism in the Pauli-Liouville basis. Numerical experiments focusing on quantum chemistry tasks demonstrate that CRLQAS outperforms existing QAS algorithms across several metrics in both noiseless and noisy environments.