# Improving thermal state preparation of Sachdev-Ye-Kitaev model with reinforcement learning on quantum hardware

arXiv: https://arxiv.org/abs/2501.11454 · HTML: https://mlsanity.com/paper/2501.11454/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Quantum reinforcement learning, one of 15 directions in Deep reinforcement learning methods.
- Builds on 17 of its 79 references that are in the map.
- First posted on arXiv 20 Jan 2025.

## Placed in

- Deep reinforcement learning methods / [Quantum reinforcement learning](https://mlsanity.com/deeprl/quantum-reinforcement-learning/)

## Builds on (most cited first, 17 of 17)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Variational Quantum Algorithms](https://mlsanity.com/paper/2012.09265/) (2020)
- [A Survey on Deep Transfer Learning](https://mlsanity.com/paper/1808.01974/) (2018)
- [Reinforcement learning for optimization of variational quantum circuit architectures](https://mlsanity.com/paper/2103.16089/) (2021)
- [Quantum Architecture Search via Deep Reinforcement Learning](https://mlsanity.com/paper/2104.07715/) (2021)
- [Curriculum reinforcement learning for quantum architecture search under hardware errors](https://mlsanity.com/paper/2402.03500/) (2024)
- [KANQAS: Kolmogorov-Arnold Network for Quantum Architecture Search](https://mlsanity.com/paper/2406.17630/) (2024)
- [Quantum Hamiltonian-Based Models and the Variational Quantum Thermalizer Algorithm](https://mlsanity.com/paper/1910.02071/) (2019)
- [Flattened Convolutional Neural Networks for Feedforward Acceleration](https://mlsanity.com/paper/1412.5474/) (2014)
- [Good Practice in CNN Feature Transfer](https://mlsanity.com/paper/1604.00133/) (2016)
- [Enhancing variational quantum state diagonalization using reinforcement learning techniques](https://mlsanity.com/paper/2306.11086/) (2023)
- [Reinforcement learning-assisted quantum architecture search for variational quantum algorithms](https://mlsanity.com/paper/2402.13754/) (2024)
- [Reinforcement Learning Assisted Recursive QAOA](https://mlsanity.com/paper/2207.06294/) (2022)
- [Reinforcement learning with learned gadgets to tackle hard quantum problems on real hardware](https://mlsanity.com/paper/2411.00230/) (2024)
- [Entanglement Diagnostics for Efficient Quantum Computation](https://mlsanity.com/paper/2102.12534/) (2021)
- [Artificially intelligent Maxwell's demon for optimal control of open quantum systems](https://mlsanity.com/paper/2408.15328/) (2024)

## Built on by (most cited first, 2 of 2)

- [BenchRL-QAS: Benchmarking reinforcement learning algorithms for quantum architecture search](https://mlsanity.com/paper/2507.12189/) (2025)
- [AlphaClifford: Efficient Clifford Synthesis and Transpilation with Model-based RL](https://mlsanity.com/paper/2608.18946/) (2026)

## Abstract (arXiv)

The Sachdev-Ye-Kitaev (SYK) model, known for its strong quantum correlations and chaotic behavior, serves as a key platform for quantum gravity studies. However, variationally preparing thermal states on near-term quantum processors for large systems (N>12, where N is the number of Majorana fermions) presents a significant challenge due to the rapid growth in the complexity of parameterized quantum circuits. This paper addresses this challenge by integrating reinforcement learning (RL) with convolutional neural networks, employing an iterative approach to optimize the quantum circuit and its parameters. The refinement process is guided by a composite reward signal derived from entropy and the expectation values of the SYK Hamiltonian. This approach reduces the number of CNOT gates by two orders of magnitude for systems N≥12 compared to traditional methods like first-order Trotterization. We demonstrate the effectiveness of the RL framework in both noiseless and noisy quantum hardware environments, maintaining high accuracy in thermal state preparation. This work advances a scalable, RL-based framework with applications for quantum gravity studies and out-of-time-ordered thermal correlators computation in quantum many-body systems on near-term quantum hardware. The code is available at https://github.com/Aqasch/solvingSYKmodelwithRL.