# Classical State Preparation for Variational Quantum Algorithms via Reinforcement Learning

arXiv: https://arxiv.org/abs/2605.23138 · HTML: https://mlsanity.com/paper/2605.23138/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Quantum reinforcement learning, one of 15 directions in Deep reinforcement learning methods.
- Builds on 18 of its 100 references that are in the map.
- First posted on arXiv 22 May 2026.

## Placed in

- Deep reinforcement learning methods / [Quantum reinforcement learning](https://mlsanity.com/deeprl/quantum-reinforcement-learning/)

## Builds on (most cited first, 18 of 18)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Variational Quantum Algorithms](https://mlsanity.com/paper/2012.09265/) (2020)
- [Barren plateaus in quantum neural network training landscapes](https://mlsanity.com/paper/1803.11173/) (2018)
- [Reinforcement Learning for Combinatorial Optimization: A Survey](https://mlsanity.com/paper/2003.03600/) (2020)
- [Cross-Entropy Loss Functions: Theoretical Analysis and Applications](https://mlsanity.com/paper/2304.07288/) (2023)
- [Horizon Reduction Makes RL Scalable](https://mlsanity.com/paper/2506.04168/) (2025)
- [Quantum Circuit Optimization with AlphaTensor](https://mlsanity.com/paper/2402.14396/) (2024)
- [Can Error Mitigation Improve Trainability of Noisy Variational Quantum Algorithms?](https://mlsanity.com/paper/2109.01051/) (2021)
- [Monte Carlo Tree Search based Hybrid Optimization of Variational Quantum Circuits](https://mlsanity.com/paper/2203.16707/) (2022)
- [CAFQA: A classical simulation bootstrap for variational quantum algorithms](https://mlsanity.com/paper/2202.12924/) (2022)
- [Optimizing the non-Clifford-count in unitary synthesis using Reinforcement Learning](https://mlsanity.com/paper/2509.21709/) (2025)
- [Performance of QAOA on Typical Instances of Constraint Satisfaction Problems with Bounded Degree](https://mlsanity.com/paper/1601.01744/) (2016)
- [Navigating the dynamic noise landscape of variational quantum algorithms with QISMET](https://mlsanity.com/paper/2209.12280/) (2022)
- [rustworkx: A High-Performance Graph Library for Python](https://mlsanity.com/paper/2110.15221/) (2021)
- [Beating the random assignment on constraint satisfaction problems of bounded degree](https://mlsanity.com/paper/1505.03424/) (2015)

## Abstract (arXiv)

Variational Quantum Algorithms (VQAs) potentially offer a pathway to practical quantum advantage, but their optimization is heavily hindered by barren plateaus and numerous local minima. While classically simulable Clifford circuits can warm-start VQAs to accelerate convergence, existing heuristic-based initialization methods struggle to scale within vast combinatorial search spaces. To overcome this bottleneck, we propose CRiSP (a Clifford Reinforcement Learning agent for State Preparation), a framework that formulates discrete prefix selection as a sequential decision-making problem. CRiSP utilizes Neural-Guided Monte Carlo Tree Search, driven by a Transformer-based policy trained via self-play, to insert learned Clifford gates before fixed parameterized rotations. This enables the construction of high-quality initial states entirely through polynomial-time classical stabilizer simulation without altering the underlying circuit architecture. By integrating a curriculum learning strategy that progressively expands the search horizon, the agent efficiently scales to deep circuits. Evaluated on QAOA benchmarks of up to 22 qubits and 1,370 parameters, CRiSP outperforms state-of-the-art Clifford initialization methods by a mean of 3.17× (max 45.02×) in average energy accuracy and 2.44× (max 16.01×) in best-achieved energy accuracy. Assessments on VQE tasks further demonstrate the framework's robustness and generalizability.