Learning the Model While Learning Q: Finite-Time Sample Complexity of Online SyncMBQ
Not on the map. Why?
Abstract (arXiv)
Reinforcement learning has witnessed significant advancements, particularly with the emergence of model-based approaches. Among these, Q-learning has proven to be a powerful algorithm in model-free settings. However, the extension of Q-learning to a model-based framework remains relatively unexplored. In this paper, we investigate the sample complexity of Q-learning when integrated with a model-based approach. The proposed algorihtms learns both the model and Q-value in an online manner. We demonstrate a near-optimal sample complexity result within a broad range of step sizes.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 12 of its 39 references that are in the map. First posted on arXiv 19 Feb 2024.
Builds on
12 of 39 references.
References · 12
- 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 748
- 2020MOReL : Model-Based Offline Reinforcement Learningcited by 582
- 2017Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuningcited by 288
- 2019Model-Based Reinforcement Learning with a Generative Model is Minimax Optimalcited by 124
- 2020Breaking the Sample Size Barrier in Model-Based Reinforcement Learning with a Generative Modelcited by 108
- 2020Finite-Time Analysis of Asynchronous Stochastic Approximation and Q-Learningcited by 93
- 2021Is Q-Learning Minimax Optimal? A Tight Sample Complexity Analysiscited by 89
- 2019Variance-reduced Q-learning is minimax optimalcited by 72
- 2019Q-learning with UCB Exploration is Sample Efficient for Infinite-Horizon MDPcited by 53
- 2020Model-Free Reinforcement Learning: from Clipped Pseudo-Regret to Sample Complexitycited by 28
- 2019A Unified Switching System Perspective and O.D.E. Analysis of Q-Learning Algorithmscited by 14
- 2014Empirical Q-Value Iterationcited by 7
Built on by
Not cited yet.