# Benchmarking Quantum Reinforcement Learning

arXiv: https://arxiv.org/abs/2501.15893 · HTML: https://mlsanity.com/paper/2501.15893/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Placed in Quantum reinforcement learning, one of 15 directions in Deep reinforcement learning methods.
- Builds on 54 of its 110 references that are in the map.
- First posted on arXiv 27 Jan 2025.

## Placed in

- Deep reinforcement learning methods / [Quantum reinforcement learning](https://mlsanity.com/deeprl/quantum-reinforcement-learning/)

## Builds on (most cited first, 30 of 54)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [DeepMind Control Suite](https://mlsanity.com/paper/1801.00690/) (2018)
- [Deep Reinforcement Learning at the Edge of the Statistical Precipice](https://mlsanity.com/paper/2108.13264/) (2021)
- [A Brief Survey of Deep Reinforcement Learning](https://mlsanity.com/paper/1708.05866/) (2017)
- [The power of quantum neural networks](https://mlsanity.com/paper/2011.00027/) (2020)
- [Benchmarking Deep Reinforcement Learning for Continuous Control](https://mlsanity.com/paper/1604.06778/) (2016)
- [A rigorous and robust quantum speed-up in supervised machine learning](https://mlsanity.com/paper/2010.02174/) (2020)
- [Reinforcement learning](https://mlsanity.com/paper/2405.10369/) (2024)
- [An Introduction to Deep Reinforcement Learning](https://mlsanity.com/paper/1811.12560/) (2018)
- [Variational Quantum Circuits for Deep Reinforcement Learning](https://mlsanity.com/paper/1907.00397/) (2019)
- [Benchmarking Model-Based Reinforcement Learning](https://mlsanity.com/paper/1907.02057/) (2019)
- [Tianshou: a Highly Modularized Deep Reinforcement Learning Library](https://mlsanity.com/paper/2107.14171/) (2021)
- [URLB: Unsupervised Reinforcement Learning Benchmark](https://mlsanity.com/paper/2110.15191/) (2021)
- [Barren Plateaus in Variational Quantum Computing](https://mlsanity.com/paper/2405.00781/) (2024)
- [Benchmarking Batch Deep Reinforcement Learning Algorithms](https://mlsanity.com/paper/1910.01708/) (2019)
- [Parametrized quantum policies for reinforcement learning](https://mlsanity.com/paper/2103.05577/) (2021)
- [Better than classical? The subtle art of benchmarking quantum machine learning models](https://mlsanity.com/paper/2403.07059/) (2024)
- [Quantum circuit architecture search for variational quantum algorithms](https://mlsanity.com/paper/2010.10217/) (2020)
- [Quantum-enhanced machine learning](https://mlsanity.com/paper/1610.08251/) (2016)
- [Reinforcement Learning with Quantum Variational Circuits](https://mlsanity.com/paper/2008.07524/) (2020)
- [Model Complexity of Deep Learning: A Survey](https://mlsanity.com/paper/2103.05127/) (2021)
- [How Many Random Seeds? Statistical Power Analysis in Deep Reinforcement Learning Experiments](https://mlsanity.com/paper/1806.08295/) (2018)
- [Empirical Design in Reinforcement Learning](https://mlsanity.com/paper/2304.01315/) (2023)
- [A Survey on Quantum Reinforcement Learning](https://mlsanity.com/paper/2211.03464/) (2022)
- [Towards a Standardised Performance Evaluation Protocol for Cooperative MARL](https://mlsanity.com/paper/2209.10485/) (2022)
- [Reinforcement Learning through Asynchronous Advantage Actor-Critic on a GPU](https://mlsanity.com/paper/1611.06256/) (2016)
- [Quantum Convolutional Neural Networks are Effectively Classically Simulable](https://mlsanity.com/paper/2408.12739/) (2024)

## Built on by (most cited first, 10 of 10)

- [Learning Encodings by Maximizing State Distinguishability: Variational Quantum Error Correction](https://mlsanity.com/paper/2506.11552/) (2025)
- [Platform-Agnostic Modular Architecture for Quantum Benchmarking](https://mlsanity.com/paper/2510.08469/) (2025)
- [Formal Verification of Noisy Quantum Reinforcement Learning Policies](https://mlsanity.com/paper/2512.01502/) (2025)
- [Quantum-Accelerated Deep Reinforcement Learning for Frequency Regulation Enhancement](https://mlsanity.com/paper/2512.04439/) (2025)
- [Quantum entanglement provides a competitive advantage in adversarial games](https://mlsanity.com/paper/2603.10289/) (2026)
- [Staged Hybridisation for Visual Quantum Reinforcement Learning via Knowledge Distillation](https://mlsanity.com/paper/2606.30520/) (2026)
- [Quantum vs. Classical Machine Learning: A Unified Empirical Comparison](https://mlsanity.com/paper/2607.01197/) (2026)
- [QANTIS: Hardware-Calibrated Sequential POMDP Belief Updates on IBM Heron](https://mlsanity.com/paper/2607.06760/) (2026)
- [SQUARE: Structured Quantum Representation Adapters as Compact Quadratic Feature Maps for Frozen Language Models](https://mlsanity.com/paper/2609.37134/) (2026)
- [A Width-Matched Comparison of Hybrid Quantum-Classical Self-Supervised Learning for Fingerprint Recognition](https://mlsanity.com/paper/2609.39172/) (2026)

## Abstract (arXiv)

Benchmarking and establishing proper statistical validation metrics for reinforcement learning (RL) remain ongoing challenges, where no consensus has been established yet. The emergence of quantum computing and its potential applications in quantum reinforcement learning (QRL) further complicate benchmarking efforts. To enable valid performance comparisons and to streamline current research in this area, we propose a novel benchmarking methodology, which is based on a statistical estimator for sample complexity and a definition of statistical outperformance. Furthermore, considering QRL, our methodology casts doubt on some previous claims regarding its superiority. We conducted experiments on a novel benchmarking environment with flexible levels of complexity. While we still identify possible advantages, our findings are more nuanced overall. We discuss the potential limitations of these results and explore their implications for empirical research on quantum advantage in QRL.