mlsanityData to 7 Oct 2026

Quantile of Means: A Bonus-Free Ensemble Method for Minimax Optimal Reinforcement Learning

arXiv 2606.20107 ↗First posted 18 Jun 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Optimal Reinforcement Learning (RL) algorithms typically rely on carefully constructed count-based uncertainty estimates to drive exploration. Although theoretically sound, such estimates are hard to compute in practical settings and therefore offer limited insight for designing exploration heuristics. Meanwhile, ensembling has emerged as a practical approach, but remains without theoretical justification. Building on a recent ensemble-based method for Multi-Armed Bandits, we propose a quantile-based ensemble method for finite-horizon Markov Decision Processes (MDPs). Our simple count-free approach achieves optimal variance-dependent regret bounds, providing theoretical grounding for ensemble-based exploration in RL.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 28 of its 35 references that are in the map. First posted on arXiv 18 Jun 2026.

Builds on

28 of 35 references.

References · 28

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  4. 2020Learning to summarize from human feedbackcited by 2,562
  5. 2015Trust Region Policy Optimizationcited by 2,503
  6. 2016Deep Exploration via Bootstrapped DQNcited by 518
  7. 2017Minimax Regret Bounds for Reinforcement Learningcited by 500
  8. 2018Is Q-learning Provably Efficient?cited by 485
  9. 2021Randomized Ensembled Double Q-Learning: Learning Fast Without a Modelcited by 278
  10. 2019Self-Supervised Exploration via Disagreementcited by 263
  11. 2013(More) Efficient Reinforcement Learning via Posterior Samplingcited by 233
  12. 2019Tighter Problem-Dependent Regret Bounds in Reinforcement Learning without Domain Knowledge using Value Function Boundscited by 194
16 more
  1. 2017Unifying PAC and Regret: Uniform PAC Bounds for Episodic Reinforcement Learningcited by 175
  2. 2020SUNRISE: A Simple Unified Framework for Ensemble Learning in Deep Reinforcement Learningcited by 150
  3. 2014Generalization and Exploration via Randomized Value Functionscited by 112
  4. 2020Episodic Reinforcement Learning in Finite MDPs: Minimax Lower Bounds Revisitedcited by 104
  5. 2017Ensemble Samplingcited by 62
  6. 2019Worst-Case Regret Bounds for Exploration via Randomized Value Functionscited by 57
  7. 2020Near-optimal Regret Bounds for Stochastic Shortest Pathcited by 50
  8. 2019Bootstrapping Upper Confidence Boundcited by 36
  9. 2018Garbage In, Reward Out: Bootstrapping Exploration in Multi-Armed Banditscited by 33
  10. 2021Ensemble Bootstrapping for Q-Learningcited by 30
  11. 2022From Dirichlet to Rubin: Optimistic Exploration in RL without Bonusescited by 22
  12. 2022Optimistic Posterior Sampling for Reinforcement Learning with Few Samples and Tight Guaranteescited by 12
  13. 2024Reinforcement Learning with Lookahead Informationcited by 8
  14. 2024Improved Regret of Linear Ensemble Samplingcited by 6
  15. 2025IL-SOAR : Imitation Learning with Soft Optimistic Actor cRiticcited by 5
  16. 2024Batch Ensemble for Variance Dependent Regret in Stochastic Banditscited by 3

Built on by

Not cited yet.