mlsanityData to 7 Oct 2026

Model-based Reinforcement Learning and the Eluder Dimension

arXiv 1406.1853 ↗First posted 7 Jun 2014stat.ML cs.LGcited by 116

Not on the map. Why?

Abstract (arXiv)

We consider the problem of learning to optimize an unknown Markov decision process (MDP). We show that, if the MDP can be parameterized within some known function class, we can obtain regret bounds that scale with the dimensionality, rather than cardinality, of the system. We characterize this dependence explicitly as tildeO(√dK dE T) where T is time elapsed, dK is the Kolmogorov dimension and dE is the eluder dimension. These represent the first unified regret bounds for model-based reinforcement learning and provide state of the art guarantees in several important settings. Moreover, we present a simple and computationally efficient algorithm posterior sampling for reinforcement learning (PSRL) that satisfies these bounds.

Key facts. Cited by 116 papers in the map (data through 7 Oct 2026). First posted on arXiv 7 Jun 2014.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 28; 2022: 20; 2023: 26; 2024: 17; 2025: 14; 2026: 10.

116 papers.

Sort:

By area:

  1. 2020Is Pessimism Provably Efficient for Offline RL?cited by 362
  2. 2021Bellman Eluder Dimension: New Rich Classes of RL Problems, and Sample-Efficient Algorithmscited by 222
  3. 2021Policy Finetuning: Bridging Sample-Efficient Offline and Online Reinforcement Learningcited by 157
  4. 2022Jump-Start Reinforcement Learningcited by 102
  5. 2021Model-free Representation Learning and Exploration in Low-rank MDPscited by 82
  6. 2022Human-in-the-loop: Provably Efficient Preference-based Reinforcement Learning with General Function Approximationcited by 80
  7. 2021Understanding Domain Randomization for Sim-to-real Transfercited by 69
  8. 2022On the Principles of Parsimony and Self-Consistency for the Emergence of Intelligencecited by 50
  9. 2021Randomized Exploration for Reinforcement Learning with General Value Function Approximationcited by 49
  10. 2021The Power of Exploiter: Provable Multi-Agent RL in Large State Spacescited by 48
  11. 2021MobILE: Model-Based Imitation Learning From Observation Alonecited by 44
  12. 2021Towards General Function Approximation in Zero-Sum Markov Gamescited by 42
38 more
  1. 2022Corruption-Robust Algorithms with Uncertainty Weighting for Nonlinear Contextual Bandits and Markov Decision Processescited by 33
  2. 2021Provable Model-based Nonlinear Bandit and Reinforcement Learning: Shelve Optimism, Embrace Virtual Curvaturecited by 32
  3. 2022A General Framework for Sample-Efficient Function Approximation in Reinforcement Learningcited by 31
  4. 2023Approximate Thompson Sampling via Epistemic Neural Networkscited by 31
  5. 2021Robust Policy Gradient against Strong Data Corruptioncited by 30
  6. 2022A Provably Efficient Model-Free Posterior Sampling Method for Episodic Reinforcement Learningcited by 30
  7. 2021Sample-Efficient Reinforcement Learning Is Feasible for Linearly Realizable MDPs with Limited Revisitingcited by 27
  8. 2021A Free Lunch from the Noise: Provable and Practical Exploration for Representation Learningcited by 24
  9. 2022Regret Bounds for Information-Directed Reinforcement Learningcited by 24
  10. 2021Representation Learning Beyond Linear Prediction Functionscited by 21
  11. 2023Efficient Exploration in Continuous-time Model-based Reinforcement Learningcited by 21
  12. 2021Understanding the Eluder Dimensioncited by 19
  13. 2022Deciding What to Model: Value-Equivalent Sampling for Reinforcement Learningcited by 19
  14. 2023Is Model Ensemble Necessary? Model-based RL via a Single Model with Lipschitz Regularized Value Functioncited by 19
  15. 2021Exploration-exploitation trade-off for continuous-time episodic reinforcement learning with linear-convex modelscited by 18
  16. 2021Online Learning for Unknown Partially Observable MDPscited by 17
  17. 2023Optimal Exploration for Model-Based RL in Nonlinear Systemscited by 17
  18. 2024Principled Preferential Bayesian Optimizationcited by 16
  19. 2021Reinforcement Learning in Reward-Mixing MDPscited by 15
  20. 2022Fast Rates in Pool-Based Batch Active Learningcited by 15
  21. 2023Provably Feedback-Efficient Reinforcement Learning via Active Reward Learningcited by 15
  22. 2025Toward Efficient Exploration by Large Language Model Agentscited by 15
  23. 2021PC-MLP: Model-based Reinforcement Learning with Policy Cover Guided Explorationcited by 14
  24. 2022A Self-Play Posterior Sampling Algorithm for Zero-Sum Markov Gamescited by 14
  25. 2022Latent Variable Representation for Reinforcement Learningcited by 14
  26. 2023Memory-Consistent Neural Networks for Imitation Learningcited by 14
  27. 2021Agnostic Reinforcement Learning with Low-Rank MDPs and Rich Observationscited by 13
  28. 2022Model-Based Reinforcement Learning with Multinomial Logistic Function Approximationcited by 13
  29. 2023On the Statistical Efficiency of Mean-Field Reinforcement Learning with General Function Approximationcited by 13
  30. 2023Provably Efficient Iterated CVaR Reinforcement Learning with Function Approximation and Human Feedbackcited by 13
  31. 2021Adaptive Discretization in Online Reinforcement Learningcited by 12
  32. 2022Bilinear Exponential Family of MDPs: Frequentist Regret Bound with Tractable Exploration and Planningcited by 12
  33. 2021Theoretically Principled Deep RL Acceleration via Nearest Neighbor Function Approximationcited by 10
  34. 2024Provably Efficient Reinforcement Learning with Multinomial Logit Function Approximationcited by 10
  35. 2022Provable Sim-to-real Transfer in Continuous Domain with Partial Observationscited by 9
  36. 2022Eluder-based Regret for Stochastic Contextual MDPscited by 9
  37. 2024Model-Based RL for Mean-Field Games is not Statistically Harder than Single-Agent RLcited by 9
  38. 2024Second Order Bounds for Contextual Bandits with Function Approximationcited by 9

50 most cited of 116 All 116 (JSON)