mlsanityData to 14 Oct 2026

Replay-buffer engineering for noise-aware quantum circuit optimization

arXiv 2604.21863 ↗First posted 23 Apr 2026quant-ph cs.AI cs.ETcited by 1

On the map: Deep reinforcement learning methods → Quantum reinforcement learning

Abstract (arXiv)

Deep reinforcement learning for quantum circuit optimization faces three bottlenecks: replay buffers that overlook temporal difference (TD) target reliability, curriculum-based architecture search requiring a full quantum-classical evaluation after every edit, and the discard of noiseless trajectories when retraining under hardware noise. We address these limitations by treating replay as a central algorithmic lever. We introduce ReaPER+, an annealed replay rule that transitions from TD-error prioritization to reliability-aware sampling as value estimates mature. ReaPER+ achieves up to 4x higher sample efficiency than fixed PER, ReaPER, and uniform replay, while matching prior on-policy solution quality with up to 32x fewer interactions At 12 qubits, fixed ReaPER reaches the lowest energy error in the fewest steps, while PER and uniform replay find more compact circuits at higher error. On tasks scaling to 20 qubits, ReaPER+ retains its advantage, demonstrating that reliability-aware annealing extends beyond small-system benchmarks. LunarLander-v3 confirms that the ReaPER+ is domain-agnostic, it improves success rates by up to 26.8% over PER and 21.8% over fixed ReaPER, with a 3% AUC gain over both. We further introduce OptCRLQAS, which amortizes quantum-classical evaluations across multiple architectural edits, reducing training wall-clock time by up to 67.5% on 12-qubit without degrading solution quality. Finally, lightweight replay-buffer transfer warm-starts noisy optimization from noiseless trajectories, without weight transfer or ε-greedy pretraining, reducing steps to chemical accuracy by 85-90% and final energy error by up to 90% relative to from-scratch learning. Transfer gains increase with system size. Together, these results establish experience storage, sampling, and transfer as decisive levers for sample efficient, noise-aware quantum circuit optimization.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Quantum reinforcement learning, one of 15 directions in Deep reinforcement learning methods. Builds on 31 of its 71 references that are in the map. First posted on arXiv 23 Apr 2026.

Builds on

31 of 71 references.

Closest ancestors · 7

Same direction, newest first

  1. 2026Reinforcement Learning for Quantum Technology26 Jan
  2. 2025BenchRL-QAS: Benchmarking reinforcement learning algorithms for quantum architecture search16 Jul
  3. 2025TensorRL-QAS: Reinforcement learning with tensor networks for improved quantum architecture search14 May
  4. 2024Reinforcement learning with learned gadgets to tackle hard quantum problems on real hardware31 Oct
  5. 2024Practical and efficient quantum circuit synthesis and transpiling with Reinforcement Learning21 May
  6. 2024Reinforcement learning-assisted quantum architecture search for variational quantum algorithms21 Feb
  7. 2024Curriculum reinforcement learning for quantum architecture search under hardware errors5 Feb

Other references · 24

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  4. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
  5. 2017Hindsight Experience Replaycited by 1,136
  6. 2015Prioritized Experience Replaycited by 1,099
  7. 2020Variational Quantum Algorithmscited by 692
  8. 2016Deep Exploration via Bootstrapped DQNcited by 527
  9. 2019A Theoretical Analysis of Deep Q-Learningcited by 215
  10. 2021Offline-to-Online Reinforcement Learning via Balanced Replay and Pessimistic Q-Ensemblecited by 214
  11. 2020Quantum circuit architecture search for variational quantum algorithmscited by 82
  12. 2021Reinforcement learning for optimization of variational quantum circuit architecturescited by 77
12 more
  1. 2024Efficient Online Reinforcement Learning Fine-Tuning Need Not Retain Offline Datacited by 63
  2. 2021Quantum Architecture Search via Deep Reinforcement Learningcited by 60
  3. 2023Challenges and Opportunities in Quantum Optimizationcited by 42
  4. 2024KANQAS: Kolmogorov-Arnold Network for Quantum Architecture Searchcited by 37
  5. 2024Quantum Circuit Optimization with AlphaTensorcited by 36
  6. 2018Reinforcement Learning Decoders for Fault-Tolerant Quantum Computationcited by 28
  7. 2022Automated Quantum Circuit Design with Nested Monte Carlo Tree Searchcited by 16
  8. 2024Prioritized Generative Replaycited by 13
  9. 2024Physics-Informed Bayesian Optimization of Variational Quantum Circuitscited by 7
  10. 2024Quantum Compiling with Reinforcement Learning on a Superconducting Processorcited by 4
  11. 2025Reliability-Adjusted Prioritized Experience Replaycited by 3
  12. 2025Adaptive Replay Buffer for Offline-to-Online Reinforcement Learningcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Generative Replay Mitigates Sample Starvation in Quantum Architecture Searchcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025FIDDLE: Reinforcement Learning for Quantum Fidelity Enhancement
  2. 2025Hybrid Reward-Driven Reinforcement Learning for Efficient Quantum Circuit Synthesis
  3. 2026Reinforcement Learning for Adaptive Composition of Quantum Circuit Optimisation Passes
  4. 2025OrQstrator: An AI-Powered Framework for Advanced Quantum Circuit Optimization
  5. 2024A Reinforcement Learning Environment for Directed Quantum Circuit Synthesis
  6. 2026Graph Reinforcement Learning for Calibration-Aware Quantum Circuit Routing
  7. 2026Beyond Reinforcement Learning: Fast and Scalable Quantum Circuit Synthesis
  8. 2024Enhancing Quantum Memory Lifetime with Measurement-Free Local Error Correction and Reinforcement Learning