# Reinforcement learning with learned gadgets to tackle hard quantum problems on real hardware

arXiv: https://arxiv.org/abs/2411.00230 · HTML: https://mlsanity.com/paper/2411.00230/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in Quantum reinforcement learning, one of 15 directions in Deep reinforcement learning methods.
- Builds on 27 of its 78 references that are in the map.
- First posted on arXiv 31 Oct 2024.

## Placed in

- Deep reinforcement learning methods / [Quantum reinforcement learning](https://mlsanity.com/deeprl/quantum-reinforcement-learning/)

## Builds on (most cited first, 27 of 27)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Diversity is All You Need: Learning Skills without a Reward Function](https://mlsanity.com/paper/1802.06070/) (2018)
- [Data-Efficient Hierarchical Reinforcement Learning](https://mlsanity.com/paper/1805.08296/) (2018)
- [The Option-Critic Architecture](https://mlsanity.com/paper/1609.05140/) (2016)
- [Variational Intrinsic Control](https://mlsanity.com/paper/1611.07507/) (2016)
- [DreamCoder: Growing generalizable, interpretable knowledge with wake-sleep Bayesian program learning](https://mlsanity.com/paper/2006.08381/) (2020)
- [Stochastic Neural Networks for Hierarchical Reinforcement Learning](https://mlsanity.com/paper/1704.03012/) (2017)
- [Meta Learning Shared Hierarchies](https://mlsanity.com/paper/1710.09767/) (2017)
- [Reinforcement learning for optimization of variational quantum circuit architectures](https://mlsanity.com/paper/2103.16089/) (2021)
- [Temporal Abstraction in Reinforcement Learning with the Successor Representation](https://mlsanity.com/paper/2110.05740/) (2021)
- [DDCO: Discovery of Deep Continuous Options for Robot Learning from Demonstrations](https://mlsanity.com/paper/1710.05421/) (2017)
- [Curriculum reinforcement learning for quantum architecture search under hardware errors](https://mlsanity.com/paper/2402.03500/) (2024)
- [KANQAS: Kolmogorov-Arnold Network for Quantum Architecture Search](https://mlsanity.com/paper/2406.17630/) (2024)
- [Quantum Circuit Optimization with AlphaTensor](https://mlsanity.com/paper/2402.14396/) (2024)
- [Quantum Architecture Search via Continual Reinforcement Learning](https://mlsanity.com/paper/2112.05779/) (2021)
- [Reinforcement learning-assisted quantum architecture search for variational quantum algorithms](https://mlsanity.com/paper/2402.13754/) (2024)
- [Optimizing Quantum Variational Circuits with Deep Reinforcement Learning](https://mlsanity.com/paper/2109.03188/) (2021)
- [Challenges for Reinforcement Learning in Quantum Circuit Design](https://mlsanity.com/paper/2312.11337/) (2023)
- [Reinforcement Learning for Variational Quantum Circuits Design](https://mlsanity.com/paper/2409.05475/) (2024)
- [Reinforcement Learning Assisted Recursive QAOA](https://mlsanity.com/paper/2207.06294/) (2022)
- [AlphaRouter: Quantum Circuit Routing with Reinforcement Learning and Tree Search](https://mlsanity.com/paper/2410.05115/) (2024)
- [QuantumCircuitOpt: An Open-source Framework for Provably Optimal Quantum Circuit Design](https://mlsanity.com/paper/2111.11674/) (2021)
- [Quantum approximate optimization via learning-based adaptive optimization](https://mlsanity.com/paper/2303.14877/) (2023)
- [How Parallel Circuit Execution Can Be Useful for NISQ Computing?](https://mlsanity.com/paper/2112.00387/) (2021)
- [An Empirical Review of Optimization Techniques for Quantum Variational Circuits](https://mlsanity.com/paper/2202.01389/) (2022)
- [A Case for Synthesis of Recursive Quantum Unitary Programs](https://mlsanity.com/paper/2311.11503/) (2023)

## Built on by (most cited first, 7 of 7)

- [TensorRL-QAS: Reinforcement learning with tensor networks for improved quantum architecture search](https://mlsanity.com/paper/2505.09371/) (2025)
- [Improving thermal state preparation of Sachdev-Ye-Kitaev model with reinforcement learning on quantum hardware](https://mlsanity.com/paper/2501.11454/) (2025)
- [BenchRL-QAS: Benchmarking reinforcement learning algorithms for quantum architecture search](https://mlsanity.com/paper/2507.12189/) (2025)
- [RhoDARTS: Differentiable Quantum Architecture Search with Density Matrix Simulations](https://mlsanity.com/paper/2506.03697/) (2025)
- [Replay-buffer engineering for noise-aware quantum circuit optimization](https://mlsanity.com/paper/2604.21863/) (2026)
- [Adversarial Effects on Expressibility and Trainability in Distributed Variational Quantum Algorithms](https://mlsanity.com/paper/2605.03629/) (2026)
- [Energy Accuracy Is Not Enough: A Structure-Aware Benchmark and Evaluation Protocol for Quantum Architecture Search](https://mlsanity.com/paper/2607.04845/) (2026)

## Abstract (arXiv)

Quantum computing offers exciting opportunities for simulating complex quantum systems and optimizing large scale combinatorial problems, but its practical use is limited by device noise and constrained connectivity. Designing quantum circuits, which are fundamental to quantum algorithms, is therefore a central challenge in current quantum hardware. Existing reinforcement learning based methods for circuit design lose accuracy when restricted to hardware native gates and device level compilation. Here, we introduce gadget reinforcement learning (GRL), which combines learning with program synthesis to automatically construct composite gates that expand the action space while respecting hardware constraints. We show that this approach improves accuracy, hardware compatibility, and scalability for transverse-field Ising and quantum chemistry problems, reaching systems of up to ten qubits within realistic computational budgets. This framework demonstrates how learned, reusable circuit building blocks can guide the co-design of algorithms and hardware for quantum processors.