# Reinforcement Learning Assisted Recursive QAOA

arXiv: https://arxiv.org/abs/2207.06294 · HTML: https://mlsanity.com/paper/2207.06294/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 48 references that are in the map.
- First posted on arXiv 13 Jul 2022.

## Builds on (most cited first, 10 of 10)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Parameterized quantum circuits as machine learning models](https://mlsanity.com/paper/1906.07682/) (2019)
- [Parametrized quantum policies for reinforcement learning](https://mlsanity.com/paper/2103.05577/) (2021)
- [Learning to Optimize Variational Quantum Circuits to Solve Combinatorial Problems](https://mlsanity.com/paper/1911.11071/) (2019)
- [Policy Gradient based Quantum Approximate Optimization Algorithm](https://mlsanity.com/paper/2002.01068/) (2020)
- [Reinforcement Learning for Many-Body Ground-State Preparation Inspired by Counterdiabatic Driving](https://mlsanity.com/paper/2010.03655/) (2020)
- [Noise-Robust End-to-End Quantum Control using Deep Autoregressive Policy Networks](https://mlsanity.com/paper/2012.06701/) (2020)
- [Low depth mechanisms for quantum optimization](https://mlsanity.com/paper/2008.08615/) (2020)
- [Classical algorithms and quantum limitations for maximum cut on high-girth graphs](https://mlsanity.com/paper/2106.05900/) (2021)
- [Bounds on approximating Max kXOR with quantum and classical local algorithms](https://mlsanity.com/paper/2109.10833/) (2021)

## Built on by (most cited first, 8 of 8)

- [KANQAS: Kolmogorov-Arnold Network for Quantum Architecture Search](https://mlsanity.com/paper/2406.17630/) (2024)
- [Enhancing variational quantum state diagonalization using reinforcement learning techniques](https://mlsanity.com/paper/2306.11086/) (2023)
- [Reinforcement learning-assisted quantum architecture search for variational quantum algorithms](https://mlsanity.com/paper/2402.13754/) (2024)
- [Reinforcement learning with learned gadgets to tackle hard quantum problems on real hardware](https://mlsanity.com/paper/2411.00230/) (2024)
- [Application of quantum-inspired generative models to small molecular datasets](https://mlsanity.com/paper/2304.10867/) (2023)
- [Improving thermal state preparation of Sachdev-Ye-Kitaev model with reinforcement learning on quantum hardware](https://mlsanity.com/paper/2501.11454/) (2025)
- [BenchRL-QAS: Benchmarking reinforcement learning algorithms for quantum architecture search](https://mlsanity.com/paper/2507.12189/) (2025)
- [Evolving Hard Maximum Cut Instances for Quantum Approximate Optimization Algorithms](https://mlsanity.com/paper/2502.12012/) (2025)

## Abstract (arXiv)

Variational quantum algorithms such as the Quantum Approximation Optimization Algorithm (QAOA) in recent years have gained popularity as they provide the hope of using NISQ devices to tackle hard combinatorial optimization problems. It is, however, known that at low depth, certain locality constraints of QAOA limit its performance. To go beyond these limitations, a non-local variant of QAOA, namely recursive QAOA (RQAOA), was proposed to improve the quality of approximate solutions. The RQAOA has been studied comparatively less than QAOA, and it is less understood, for instance, for what family of instances it may fail to provide high quality solutions. However, as we are tackling mathsfNP-hard problems (specifically, the Ising spin model), it is expected that RQAOA does fail, raising the question of designing even better quantum algorithms for combinatorial optimization. In this spirit, we identify and analyze cases where RQAOA fails and, based on this, propose a reinforcement learning enhanced RQAOA variant (RL-RQAOA) that improves upon RQAOA. We show that the performance of RL-RQAOA improves over RQAOA: RL-RQAOA is strictly better on these identified instances where RQAOA underperforms, and is similarly performing on instances where RQAOA is near-optimal. Our work exemplifies the potentially beneficial synergy between reinforcement learning and quantum (inspired) optimization in the design of new, even better heuristics for hard problems.