mlsanityData to 7 Oct 2026

Improved Model-based Reinforcement Learning with Smooth Kernels

arXiv 2605.07218 ↗First posted 8 May 2026cs.LG stat.MLcited by 0

Not on the map. Why?

Abstract (arXiv)

For continuous state-action space scenarios, classical reinforcement learning (RL) theory predominantly focuses on low-rank Markov decision processes (MDPs), which provide sample-efficient guarantees at the expense of restrictive structural assumptions. Kernel smoothing model-based approaches offer a promising alternative paradigm that instead leverages the smoothness of the MDP and employs non-parametric kernel smoothing estimates of transition dynamics. This paper proposes a new kernel-smoothing model-based approach for online reinforcement learning in finite-horizon settings under Lipschitz continuity assumptions on the MDP. By incorporating a Bernstein-style exploration bonus into the kernel smoothing framework, our method achieves a regret bound which improves upon the state-of-the-art regret bound in its dependence on the horizon. The theoretical advancement relies on a delicate analysis of the synergy between Bernstein-style bonuses and kernel smoothing, where a new tight Bernstein-type concentration inequality for martingales may be of independent interest.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 32 of its 55 references that are in the map. First posted on arXiv 8 May 2026.

Builds on

32 of 55 references.

References · 32

  1. 2019Provably Efficient Reinforcement Learning with Linear Function Approximationcited by 514
  2. 2017Minimax Regret Bounds for Reinforcement Learningcited by 500
  3. 2018Is Q-learning Provably Efficient?cited by 485
  4. 2021Bridging Offline Reinforcement Learning and Imitation Learning: A Tale of Pessimismcited by 275
  5. 2020Model-Based Reinforcement Learning with Value-Targeted Regressioncited by 233
  6. 2021Policy Finetuning: Bridging Sample-Efficient Offline and Online Reinforcement Learningcited by 157
  7. 2020Is Reinforcement Learning More Difficult Than Bandits? A Near-optimal Algorithm Escaping the Curse of Horizoncited by 99
  8. 2020Information Theoretic Regret Bounds for Online Nonlinear Controlcited by 87
  9. 2022Pessimistic Q-Learning for Offline Reinforcement Learning: Towards Optimal Sample Complexitycited by 84
  10. 2022Settling the Sample Complexity of Model-Based Offline Reinforcement Learningcited by 82
  11. 2017Lower Bounds on Regret for Noisy Gaussian Process Bandit Optimizationcited by 77
  12. 2022Nearly Minimax Optimal Reinforcement Learning for Linear Markov Decision Processescited by 57
20 more
  1. 2022Computationally Efficient Horizon-Free Reinforcement Learning for Linear Mixture MDPscited by 52
  2. 2022VOQL: Towards Optimal Regret in Model-free RL with Nonlinear Function Approximationcited by 44
  3. 2023Settling the Sample Complexity of Online Reinforcement Learningcited by 44
  4. 2022The Efficacy of Pessimism in Asynchronous Q-Learningcited by 41
  5. 2020A Kernel-Based Approach to Non-Stationary Reinforcement Learning in Metric Spacescited by 32
  6. 2020Model-Free Reinforcement Learning: from Clipped Pseudo-Regret to Sample Complexitycited by 28
  7. 2022Horizon-Free Reinforcement Learning in Polynomial Time: the Power of Stationary Policiescited by 27
  8. 2020Kernel-Based Reinforcement Learning: A Finite-Time Analysiscited by 23
  9. 2021Implicit Finite-Horizon Approximation and Efficient Optimal Algorithms for Stochastic Shortest Pathcited by 23
  10. 2023Kernelized Reinforcement Learning with Order Optimal Regret Boundscited by 20
  11. 2019Efficient Model-free Reinforcement Learning in Metric Spacescited by 18
  12. 2019Adaptive Discretization for Episodic Reinforcement Learning in Metric Spacescited by 16
  13. 2014Practical Kernel-Based Reinforcement Learningcited by 14
  14. 2020Adaptive Discretization for Model-Based Reinforcement Learningcited by 13
  15. 2024Achieving Tractable Minimax Optimal Regret in Average Reward MDPscited by 13
  16. 2023Sample Complexity of Kernel-Based Q-Learningcited by 8
  17. 2020Value Function Approximations via Kernel Embeddings for No-Regret Reinforcement Learningcited by 7
  18. 2024Horizon-Free Regret for Linear Markov Decision Processescited by 6
  19. 2024Leveraging Unlabeled Data Sharing through Kernel Function Approximation in Offline Reinforcement Learningcited by 2
  20. 2025Near-Optimal Sample Complexity in Reward-Free Kernel-Based Reinforcement Learningcited by 1

Built on by

Not cited yet.