# Monte Carlo Tree Search based Hybrid Optimization of Variational Quantum Circuits

arXiv: https://arxiv.org/abs/2203.16707 · HTML: https://mlsanity.com/paper/2203.16707/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 95 references that are in the map.
- First posted on arXiv 30 Mar 2022.

## Builds on (most cited first, 29 of 29)

- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [Neural Architecture Search with Reinforcement Learning](https://mlsanity.com/paper/1611.01578/) (2016)
- [Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model](https://mlsanity.com/paper/1911.08265/) (2019)
- [Soft Actor-Critic Algorithms and Applications](https://mlsanity.com/paper/1812.05905/) (2018)
- [Variational Quantum Algorithms](https://mlsanity.com/paper/2012.09265/) (2020)
- [ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware](https://mlsanity.com/paper/1812.00332/) (2018)
- [Maximum a Posteriori Policy Optimisation](https://mlsanity.com/paper/1806.06920/) (2018)
- [Mastering Atari Games with Limited Data](https://mlsanity.com/paper/2111.00210/) (2021)
- [AutoML-Zero: Evolving Machine Learning Algorithms From Scratch](https://mlsanity.com/paper/2003.03384/) (2020)
- [QuantumNAS: Noise-Adaptive Search for Robust Quantum Circuits](https://mlsanity.com/paper/2107.10845/) (2021)
- [Learning Search Space Partition for Black-box Optimization using Monte Carlo Tree Search](https://mlsanity.com/paper/2007.00708/) (2020)
- [Conservative Objective Models for Effective Offline Model-Based Optimization](https://mlsanity.com/paper/2107.06882/) (2021)
- [Simple random search provides a competitive approach to reinforcement learning](https://mlsanity.com/paper/1803.07055/) (2018)
- [Quantum Architecture Search via Deep Reinforcement Learning](https://mlsanity.com/paper/2104.07715/) (2021)
- [Self-Paced Deep Reinforcement Learning](https://mlsanity.com/paper/2004.11812/) (2020)
- [Universal Quantum Control through Deep Reinforcement Learning](https://mlsanity.com/paper/1803.01857/) (2018)
- [Continuous-Discrete Reinforcement Learning for Hybrid Control in Robotics](https://mlsanity.com/paper/2001.00449/) (2020)
- [Discrete and Continuous Action Representation for Practical RL in Video Games](https://mlsanity.com/paper/1912.11077/) (2019)
- [Optimizing Quantum Error Correction Codes with Reinforcement Learning](https://mlsanity.com/paper/1812.08451/) (2018)
- [Enabling Multi-programming Mechanism for Quantum Computing in the NISQ Era](https://mlsanity.com/paper/2102.05321/) (2021)
- [Natural Evolutionary Strategies for Variational Quantum Computation](https://mlsanity.com/paper/2012.00101/) (2020)
- [Policy Gradient based Quantum Approximate Optimization Algorithm](https://mlsanity.com/paper/2002.01068/) (2020)
- [Fidelity-based Probabilistic Q-learning for Control of Quantum Systems](https://mlsanity.com/paper/1806.03145/) (2018)
- [Measurement-based adaptation protocol with quantum reinforcement learning](https://mlsanity.com/paper/1803.05340/) (2018)
- [Noise-Robust End-to-End Quantum Control using Deep Autoregressive Policy Networks](https://mlsanity.com/paper/2012.06701/) (2020)
- [Active Learning Approach to Optimization of Experimental Control](https://mlsanity.com/paper/2003.11804/) (2020)
- [Taking gradients through experiments: LSTMs and memory proximal policy optimization for black-box quantum control](https://mlsanity.com/paper/1802.04063/) (2018)
- [Glassy Phase of Optimal Quantum Control](https://mlsanity.com/paper/1803.10856/) (2018)

## Built on by (most cited first, 10 of 10)

- [Quantum Circuit Optimization with AlphaTensor](https://mlsanity.com/paper/2402.14396/) (2024)
- [Quantum Architecture Search: A Survey](https://mlsanity.com/paper/2406.06210/) (2024)
- [Differentiable Quantum Architecture Search in Asynchronous Quantum Reinforcement Learning](https://mlsanity.com/paper/2407.18202/) (2024)
- [Quantum Machine Learning Architecture Search via Deep Reinforcement Learning](https://mlsanity.com/paper/2407.20147/) (2024)
- ['Eliv'agar: Efficient Quantum Circuit Search for Classification](https://mlsanity.com/paper/2401.09393/) (2024)
- [Evolutionary Optimization for Designing Variational Quantum Circuits with High Model Capacity](https://mlsanity.com/paper/2412.12484/) (2024)
- [QuACK: Accelerating Gradient-Based Quantum Optimization with Koopman Operator Learning](https://mlsanity.com/paper/2211.01365/) (2022)
- [Decoding surface codes with deep reinforcement learning and probabilistic policy reuse](https://mlsanity.com/paper/2212.11890/) (2022)
- [Quantum Architecture Search for Solving Quantum Machine Learning Tasks](https://mlsanity.com/paper/2509.11198/) (2025)
- [Classical State Preparation for Variational Quantum Algorithms via Reinforcement Learning](https://mlsanity.com/paper/2605.23138/) (2026)

## Abstract (arXiv)

Variational quantum algorithms stand at the forefront of simulations on near-term and future fault-tolerant quantum devices. While most variational quantum algorithms involve only continuous optimization variables, the representational power of the variational ansatz can sometimes be significantly enhanced by adding certain discrete optimization variables, as is exemplified by the generalized quantum approximate optimization algorithm (QAOA). However, the hybrid discrete-continuous optimization problem in the generalized QAOA poses a challenge to the optimization. We propose a new algorithm called MCTS-QAOA, which combines a Monte Carlo tree search method with an improved natural policy gradient solver to optimize the discrete and continuous variables in the quantum circuit, respectively. We find that MCTS-QAOA has excellent noise-resilience properties and outperforms prior algorithms in challenging instances of the generalized QAOA.