mlsanityData to 7 Oct 2026

Data-Driven Online Model Selection With Regret Guarantees

arXiv 2306.02869 ↗First posted 5 Jun 2023cs.LG cs.AI stat.MLcited by 11

Not on the map. Why?

Abstract (arXiv)

We consider model selection for sequential decision making in stochastic environments with bandit feedback, where a meta-learner has at its disposal a pool of base learners, and decides on the fly which action to take based on the policies recommended by each base learner. Model selection is performed by regret balancing but, unlike the recent literature on this subject, we do not assume any prior knowledge about the base learners like candidate regret guarantees; instead, we uncover these quantities in a data-driven manner. The meta-learner is therefore able to leverage the realized regret incurred by each base learner for the learning environment at hand (as opposed to the expected regret), and single out the best such regret. We design two model selection algorithms operating with this more ambitious notion of regret and, besides proving model selection guarantees via regret balancing, we experimentally demonstrate the compelling practical benefits of dealing with actual regrets instead of candidate regret bounds.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 14 of its 33 references that are in the map. First posted on arXiv 5 Jun 2023.

Builds on

14 of 33 references.

References · 14

  1. 2016Corralling a Band of Bandit Algorithmscited by 99
  2. 2021Adapting to Misspecification in Contextual Banditscited by 81
  3. 2020Model Selection in Contextual Stochastic Bandit Problemscited by 71
  4. 2019Model selection for contextual banditscited by 60
  5. 2020Regret Bound Balancing and Elimination for Model Selection in Bandits and RLcited by 45
  6. 2021A Model Selection Approach for Corruption Robust Reinforcement Learningcited by 45
  7. 2019OSOM: A simultaneously optimal algorithm for multi-armed and linear contextual banditscited by 28
  8. 2020Online Model Selection for Reinforcement Learning with Function Approximationcited by 28
  9. 2020Problem-Complexity Adaptive Model Selection for Stochastic Linear Banditscited by 26
  10. 2021The Pareto Frontier of model selection for general Contextual Banditscited by 24
  11. 2020Corralling Stochastic Bandit Algorithmscited by 23
  12. 2020Regret Balancing for Bandit and RL Model Selectioncited by 20
2 more
  1. 2022Best of Both Worlds Model Selectioncited by 11
  2. 2020Rate-adaptive model selection over a collection of black-box contextual bandit algorithmscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 5; 2026: 3.

11 papers.

  1. 2024Efficient Sequential Decision Making with Large Language Modelscited by 16
  2. 2025BOAD: Discovering Hierarchical Software Engineering Agents via Bandit Optimizationcited by 7
  3. 2026Reinforcement Learning in the Real World: A Survey of Statistical Challenges and Future Directionscited by 5
  4. 2024Symmetric Linear Bandits with Hidden Symmetrycited by 1
  5. 2025Managing Self-Learning Experts under Per-Round Budget Constraintscited by 1
  6. 2024Learning Rate-Free Reinforcement Learning: A Case for Model Selection with Non-Stationary Objectivescited by 0
  7. 2025Complexity as Advantage: A Regret-Based Perspective on Emergent Structurecited by 0
  8. 2025Empirical Comparison of Forgetting Mechanisms for UCB-based Algorithms on a Data-Driven Simulation Platformcited by 0
  9. 2025Improved Training Mechanism for Reinforcement Learning via Online Model Selectioncited by 0
  10. 2026Bayesian Online Model Selectioncited by 0
  11. 2026Beyond Static Bias: Adaptive Multi-Fidelity Bandits with Improving Proxiescited by 0