# TensorRL-QAS: Reinforcement learning with tensor networks for improved quantum architecture search

arXiv: https://arxiv.org/abs/2505.09371 · HTML: https://mlsanity.com/paper/2505.09371/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in Quantum reinforcement learning, one of 15 directions in Deep reinforcement learning methods.
- Builds on 27 of its 82 references that are in the map.
- First posted on arXiv 14 May 2025.

## Placed in

- Deep reinforcement learning methods / [Quantum reinforcement learning](https://mlsanity.com/deeprl/quantum-reinforcement-learning/)

## Builds on (most cited first, 27 of 27)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Dueling Network Architectures for Deep Reinforcement Learning](https://mlsanity.com/paper/1511.06581/) (2015)
- [Variational Quantum Algorithms](https://mlsanity.com/paper/2012.09265/) (2020)
- [Barren Plateaus in Variational Quantum Computing](https://mlsanity.com/paper/2405.00781/) (2024)
- [QuantumNAS: Noise-Adaptive Search for Robust Quantum Circuits](https://mlsanity.com/paper/2107.10845/) (2021)
- [Quantum circuit architecture search for variational quantum algorithms](https://mlsanity.com/paper/2010.10217/) (2020)
- [Reinforcement learning for optimization of variational quantum circuit architectures](https://mlsanity.com/paper/2103.16089/) (2021)
- [Truly Proximal Policy Optimization](https://mlsanity.com/paper/1903.07940/) (2019)
- [Exploiting symmetry in variational quantum machine learning](https://mlsanity.com/paper/2205.06217/) (2022)
- [Quantum Architecture Search via Deep Reinforcement Learning](https://mlsanity.com/paper/2104.07715/) (2021)
- [Group-Invariant Quantum Machine Learning](https://mlsanity.com/paper/2205.02261/) (2022)
- [Equivariant quantum circuits for learning on weighted graphs](https://mlsanity.com/paper/2205.06109/) (2022)
- [Curriculum reinforcement learning for quantum architecture search under hardware errors](https://mlsanity.com/paper/2402.03500/) (2024)
- [Quantum Circuit Optimization with AlphaTensor](https://mlsanity.com/paper/2402.14396/) (2024)
- [Quantum Architecture Search: A Survey](https://mlsanity.com/paper/2406.06210/) (2024)
- [Quantum Architecture Search via Continual Reinforcement Learning](https://mlsanity.com/paper/2112.05779/) (2021)
- [Tensor networks for quantum machine learning](https://mlsanity.com/paper/2303.11735/) (2023)
- [A semi-agnostic ansatz with variable structure for quantum machine learning](https://mlsanity.com/paper/2103.06712/) (2021)
- [Enhancing variational quantum state diagonalization using reinforcement learning techniques](https://mlsanity.com/paper/2306.11086/) (2023)
- [Reinforcement learning-assisted quantum architecture search for variational quantum algorithms](https://mlsanity.com/paper/2402.13754/) (2024)
- [Optimizing Quantum Variational Circuits with Deep Reinforcement Learning](https://mlsanity.com/paper/2109.03188/) (2021)
- [Reinforcement learning with learned gadgets to tackle hard quantum problems on real hardware](https://mlsanity.com/paper/2411.00230/) (2024)
- [Circuit Transformations for Quantum Architectures](https://mlsanity.com/paper/1902.09102/) (2019)

## Built on by (most cited first, 7 of 7)

- [Beyond Reinforcement Learning: Fast and Scalable Quantum Circuit Synthesis](https://mlsanity.com/paper/2602.15146/) (2026)
- [Replay-buffer engineering for noise-aware quantum circuit optimization](https://mlsanity.com/paper/2604.21863/) (2026)
- [Differentiable Logical Programming for Quantum Circuit Discovery and Optimization](https://mlsanity.com/paper/2602.08880/) (2026)
- [AlphaClifford: Efficient Clifford Synthesis and Transpilation with Model-based RL](https://mlsanity.com/paper/2608.18946/) (2026)
- [Modeling quantum neural network gradient with reinforcement learning](https://mlsanity.com/paper/2609.31066/) (2026)
- [QuPID: Quantum Parameter-Efficient Input-Dependent Retrieval Adaptation for Medical RAG](https://mlsanity.com/paper/2609.33351/) (2026)
- [SQUARE: Structured Quantum Representation Adapters as Compact Quadratic Feature Maps for Frozen Language Models](https://mlsanity.com/paper/2609.37134/) (2026)

## Abstract (arXiv)

Variational quantum algorithms hold the promise to address meaningful quantum problems already on noisy intermediate-scale quantum hardware. In spite of the promise, they face the challenge of designing quantum circuits that both solve the target problem and comply with device limitations. Quantum architecture search (QAS) automates the design process of quantum circuits, with reinforcement learning (RL) emerging as a promising approach. Yet, RL-based QAS methods encounter significant scalability issues, as computational and training costs grow rapidly with the number of qubits, circuit depth, and hardware noise. To address these challenges, we introduce textitTensorRL-QAS, an improved framework that combines tensor network methods with RL for QAS. By warm-starting the QAS with a matrix product state approximation of the target solution, TensorRL-QAS effectively narrows the search space to physically meaningful circuits and accelerates the convergence to the desired solution. Tested on several quantum chemistry problems of up to 12-qubit, TensorRL-QAS achieves up to a 10-fold reduction in CNOT count and circuit depth compared to baseline methods, while maintaining or surpassing chemical accuracy. It reduces classical optimizer function evaluation by up to 100-fold, accelerates training episodes by up to 98%, and can achieve 50% success probability for 10-qubit systems, far exceeding the <1% rates of baseline. Robustness and versatility are demonstrated both in the noiseless and noisy scenarios, where we report a simulation of an 8-qubit system. Furthermore, TensorRL-QAS demonstrates effectiveness on systems on 20-qubit quantum systems, positioning it as a state-of-the-art quantum circuit discovery framework for near-term hardware and beyond.