Quantum-Inspired Episode Selection for Monte Carlo Reinforcement Learning via QUBO Optimization
On the map: Deep reinforcement learning methods → Quantum reinforcement learning
Abstract (arXiv)
Monte Carlo (MC) reinforcement learning suffers from high sample complexity, especially in environments with sparse rewards, large state spaces, and correlated trajectories. We address these limitations by reformulating episode selection as a Quadratic Unconstrained Binary Optimization (QUBO) problem and solving it with quantum-inspired samplers. Our method, MC+QUBO, integrates a combinatorial filtering step into standard MC policy evaluation: from each batch of trajectories, we select a subset that maximizes cumulative reward while promoting state-space coverage. This selection is encoded as a QUBO, where linear terms favor high-reward episodes and quadratic terms penalize redundancy. We explore both Simulated Quantum Annealing (SQA) and Simulated Bifurcation (SB) as black-box solvers within this framework. Experiments in a finite-horizon GridWorld demonstrate that MC+QUBO outperforms vanilla MC in convergence speed and final policy quality, highlighting the potential of quantum-inspired optimization as a decision-making subroutine in reinforcement learning.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Quantum reinforcement learning, one of 15 directions in Deep reinforcement learning methods. Builds on 3 of its 20 references that are in the map. First posted on arXiv 24 Jan 2026.
Builds on
3 of 20 references.
References · 3
- 2016Simulated Quantum Annealing Can Be Exponentially Faster than Classical Simulated Annealingcited by 11
- 2020On the Convergence of Reinforcement Learning with Monte Carlo Exploring Startscited by 9
- 2023On The Convergence Of Policy Iteration-Based Reinforcement Learning With Monte Carlo Policy Evaluationcited by 9
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2021Quantum Algorithms for Reinforcement Learning with a Generative Model
- 2026Quantum Annealing Enhanced Reinforcement Learning for Accurate Remaining Useful Lifetime Prediction
- 2026Improved Quantum Algorithms for Reinforcement Learning Under a Generative Model
- 2025Quantum Bayesian Networks Can Speed up Reinforcement Learning in Partially Observable Environments
- 2026QuantFPFlow: Quantum Amplitude Estimation for Fokker--Planck Policy Optimisation in Continuous Reinforcement Learning
- 2024Hamiltonian-based Quantum Reinforcement Learning for Neural Combinatorial Optimization
- 2026Hybrid Quantum Reinforcement Learning with QAOA for Improved Vehicle Routing Optimization
- 2025ARDNS-FN-Quantum: A Quantum-Enhanced Reinforcement Learning Framework with Cognitive-Inspired Adaptive Exploration for Dynamic Environments