mlsanityData to 14 Oct 2026

Improved Training Mechanism for Reinforcement Learning via Online Model Selection

arXiv 2512.02214 ↗First posted 1 Dec 2025cs.LG cs.AIcited by 0

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

We study the problem of online model selection in reinforcement learning, where the selector has access to a class of reinforcement learning agents and learns to adaptively select the agent with the right configuration. Our goal is to establish the improved efficiency and performance gains achieved by integrating online model selection methods into reinforcement learning training procedures. We examine the theoretical characterizations that are effective for identifying the right configuration in practice, and address three practical criteria from a theoretical perspective: 1) Efficient resource allocation, 2) Adaptation under non-stationary dynamics, and 3) Training stability across different seeds. Our theoretical results are accompanied by empirical evidence from various model selection tasks in reinforcement learning, including neural architecture selection, step-size selection, and self model selection.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 13 of its 22 references that are in the map. First posted on arXiv 1 Dec 2025.

Builds on

13 of 22 references.

References · 13

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2016Corralling a Band of Bandit Algorithmscited by 100
  3. 2021Adapting to Misspecification in Contextual Banditscited by 81
  4. 2020Model Selection in Contextual Stochastic Bandit Problemscited by 71
  5. 2022Automated Reinforcement Learning (AutoRL): A Survey and Open Problemscited by 70
  6. 2020Regret Bound Balancing and Elimination for Model Selection in Bandits and RLcited by 47
  7. 2023Foundations of Reinforcement Learning and Interactive Decision Makingcited by 33
  8. 2021The Pareto Frontier of model selection for general Contextual Banditscited by 24
  9. 2022Oracle Inequalities for Model Selection in Offline Reinforcement Learningcited by 13
  10. 2023Data-Driven Online Model Selection With Regret Guaranteescited by 11
  11. 2017Online Meta-learning by Parallel Algorithm Competitioncited by 9
  12. 2023Anytime Model Selection in Linear Banditscited by 6
1 more
  1. 2025Model Selection for Off-policy Evaluation: New Algorithms and Experimental Protocolcited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Learning Rate-Free Reinforcement Learning: A Case for Model Selection with Non-Stationary Objectives
  2. 2025Enhancing Offline Model-Based RL via Active Model Selection: A Bayesian Optimization Perspective
  3. 2022MOORe: Model-based Offline-to-Online Reinforcement Learning
  4. 2023Model-based Offline Reinforcement Learning with Local Misspecification
  5. 2025Where to Intervene: Action Selection in Deep Reinforcement Learning
  6. 2025Behavior-Adaptive Q-Learning: A Unifying Framework for Offline-to-Online RL
  7. 2026Uncertainty-Aware Selection of Online Algorithms with Simulator Ensembles
  8. 2025Efficient Model-Based Reinforcement Learning for Robot Control via Online Optimization