# Dissecting Quantum Reinforcement Learning: A Systematic Evaluation of Key Components

arXiv: https://arxiv.org/abs/2511.17112 · HTML: https://mlsanity.com/paper/2511.17112/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Quantum reinforcement learning, one of 15 directions in Deep reinforcement learning methods.
- Builds on 18 of its 35 references that are in the map.
- First posted on arXiv 21 Nov 2025.

## Placed in

- Deep reinforcement learning methods / [Quantum reinforcement learning](https://mlsanity.com/deeprl/quantum-reinforcement-learning/)

## Builds on (most cited first, 18 of 18)

- [Variational Quantum Algorithms](https://mlsanity.com/paper/2012.09265/) (2020)
- [The effect of data encoding on the expressive power of variational quantum machine learning models](https://mlsanity.com/paper/2008.08605/) (2020)
- [Connecting ansatz expressibility to gradient magnitudes and barren plateaus](https://mlsanity.com/paper/2101.02138/) (2021)
- [Variational Quantum Circuits for Deep Reinforcement Learning](https://mlsanity.com/paper/1907.00397/) (2019)
- [Parametrized quantum policies for reinforcement learning](https://mlsanity.com/paper/2103.05577/) (2021)
- [Quantum reinforcement learning in continuous action space](https://mlsanity.com/paper/2012.10711/) (2020)
- [Quantum Policy Gradient Algorithm with Optimized Action Decoding](https://mlsanity.com/paper/2212.06663/) (2022)
- [Quantum Deep Reinforcement Learning for Robot Navigation Tasks](https://mlsanity.com/paper/2202.12180/) (2022)
- [Automated Quantum Circuit Design with Nested Monte Carlo Tree Search](https://mlsanity.com/paper/2207.00132/) (2022)
- [Policy Gradients using Variational Quantum Circuits](https://mlsanity.com/paper/2203.10591/) (2022)
- [VQC-Based Reinforcement Learning with Data Re-uploading: Performance and Trainability](https://mlsanity.com/paper/2401.11555/) (2024)
- [Quantum Advantage Actor-Critic for Reinforcement Learning](https://mlsanity.com/paper/2401.07043/) (2024)
- [Variational Quantum Soft Actor-Critic for Robotic Arm Control](https://mlsanity.com/paper/2212.11681/) (2022)
- [A Study on Optimization Techniques for Variational Quantum Circuits in Reinforcement Learning](https://mlsanity.com/paper/2405.12354/) (2024)
- [Trainability issues in quantum policy gradients](https://mlsanity.com/paper/2406.09614/) (2024)
- [Robustness and Generalization in Quantum Reinforcement Learning via Lipschitz Regularization](https://mlsanity.com/paper/2410.21117/) (2024)
- [Hamiltonian-based Quantum Reinforcement Learning for Neural Combinatorial Optimization](https://mlsanity.com/paper/2405.07790/) (2024)
- [A quantum-classical reinforcement learning model to play Atari games](https://mlsanity.com/paper/2412.08725/) (2024)

## Built on by (most cited first, 2 of 2)

- [Quantum entanglement provides a competitive advantage in adversarial games](https://mlsanity.com/paper/2603.10289/) (2026)
- [Staged Hybridisation for Visual Quantum Reinforcement Learning via Knowledge Distillation](https://mlsanity.com/paper/2606.30520/) (2026)

## Abstract (arXiv)

Parameterised quantum circuit (PQC) based Quantum Reinforcement Learning (QRL) has emerged as a promising paradigm at the intersection of quantum computing and reinforcement learning (RL). By design, PQCs create hybrid quantum-classical models, but their practical applicability remains uncertain due to training instabilities, barren plateaus (BPs), and the difficulty of isolating the contribution of individual pipeline components. In this work, we dissect PQC based QRL architectures through a systematic experimental evaluation of three aspects recurrently identified as critical: (i) data embedding strategies, with Data Reuploading (DR) as an advanced approach; (ii) ansatz design, particularly the role of entanglement; and (iii) post-processing blocks after quantum measurement, with a focus on the underexplored Output Reuse (OR) technique. Using a unified PPO-CartPole framework, we perform controlled comparisons between hybrid and classical agents under identical conditions. Our results show that OR, though purely classical, exhibits distinct behaviour in hybrid pipelines, that DR improves trainability and stability, and that stronger entanglement can degrade optimisation, offsetting classical gains. Together, these findings provide controlled empirical evidence of the interplay between quantum and classical contributions, and establish a reproducible framework for systematic benchmarking and component-wise analysis in QRL.