mlsanityData to 7 Oct 2026

PC-MLP: Model-based Reinforcement Learning with Policy Cover Guided Exploration

arXiv 2107.07410 ↗First posted 15 Jul 2021cs.LGcited by 14

Not on the map. Why?

Abstract (arXiv)

Model-based Reinforcement Learning (RL) is a popular learning paradigm due to its potential sample efficiency compared to model-free RL. However, existing empirical model-based RL approaches lack the ability to explore. This work studies a computationally and statistically efficient model-based algorithm for both Kernelized Nonlinear Regulators (KNR) and linear Markov Decision Processes (MDPs). For both models, our algorithm guarantees polynomial sample complexity and only uses access to a planning oracle. Experimentally, we first demonstrate the flexibility and efficacy of our algorithm on a set of exploration challenging control tasks where existing empirical model-based RL approaches completely fail. We then show that our approach retains excellent performance even in common dense reward control benchmarks that do not require heavy exploration. Finally, we demonstrate that our method can also perform reward-free exploration efficiently. Our code can be found at https://github.com/yudasong/PCMLP.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 33 of its 55 references that are in the map. First posted on arXiv 15 Jul 2021.

Builds on

33 of 55 references.

References · 33

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  2. 2015Trust Region Policy Optimizationcited by 2,503
  3. 2016OpenAI Gymcited by 1,684
  4. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  5. 2018Exploration by Random Network Distillationcited by 966
  6. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 753
  7. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 748
  8. 2019Model-Based Reinforcement Learning for Ataricited by 495
  9. 2017Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuningcited by 288
  10. 2018Multi-Goal Reinforcement Learning: Challenging Robotics Environments and Request for Researchcited by 256
  11. 2020Model-Based Reinforcement Learning with Value-Targeted Regressioncited by 233
  12. 2020FLAMBE: Structural Complexity and Representation Learning of Low Rank MDPscited by 232
21 more
  1. 2018Model-Ensemble Trust-Region Policy Optimizationcited by 214
  2. 2019Reinforcement Learning in Feature Space: Matrix Bandit, Kernels, and Regret Boundcited by 199
  3. 2019Benchmarking Model-Based Reinforcement Learningcited by 158
  4. 2017A General Safety Framework for Learning-Based Control in Uncertain Robotic Systemscited by 150
  5. 2018Algorithmic Framework for Model-based Deep Reinforcement Learning with Theoretical Guaranteescited by 138
  6. 2021Provably Efficient Reinforcement Learning with Linear Function Approximation Under Adaptivity Constraintscited by 124
  7. 2014Model-based Reinforcement Learning and the Eluder Dimensioncited by 116
  8. 2020Naive Exploration is Optimal for Online LQRcited by 111
  9. 2020Provably Efficient Reinforcement Learning for Discounted MDPs with Feature Mappingcited by 102
  10. 2019Sample Complexity of Reinforcement Learning using Linearly Combined Model Ensemblescited by 100
  11. 2020PC-PG: Policy Cover Directed Exploration for Provable Policy Gradient Learningcited by 91
  12. 2020Information Theoretic Regret Bounds for Online Nonlinear Controlcited by 87
  13. 2018Regret Bounds for Robust Adaptive Control of the Linear Quadratic Regulatorcited by 86
  14. 2020Active Learning for Nonlinear System Identification with Guaranteescited by 78
  15. 2019Keeping Your Distance: Solving Sparse Reward Tasks Using Self-Balancing Shaped Rewardscited by 76
  16. 2020A Game Theoretic Framework for Model Based Reinforcement Learningcited by 72
  17. 2019Learning Linear-Quadratic Regulators Efficiently with only √T Regretcited by 67
  18. 2019Information-Theoretic Confidence Bounds for Reinforcement Learningcited by 51
  19. 2018Dual Policy Iterationcited by 22
  20. 2017Goal-Driven Dynamics Learning via Bayesian Optimizationcited by 19
  21. 2019Certainty Equivalence is Efficient for Linear Quadratic Controlcited by 13

Built on by

Cited by year

Citing papers per year: 2021: 2; 2022: 1; 2023: 3; 2024: 5; 2025: 3.

14 papers.

Sort:

By area:

  1. 2021Representation Learning for Online and Offline RL in Low-rank MDPscited by 128
  2. 2025Outcome-based Exploration for LLM Reasoningcited by 79
  3. 2023Optimal Exploration for Model-Based RL in Nonlinear Systemscited by 17
  4. 2023The Virtues of Laziness in Model-based RL: A Unified Objective and Algorithmscited by 12
  5. 2024Model-based RL as a Minimalist Approach to Horizon-Free and Second-Order Boundscited by 11
  6. 2024Active Learning for Control-Oriented Identification of Nonlinear Systemscited by 7
  7. 2024Leveraging Approximate Model-based Shielding for Probabilistic Safety Guarantees in Continuous Environmentscited by 5
  8. 2024Double Duality: Variational Primal-Dual Policy Optimization for Constrained Reinforcement Learningcited by 2
  9. 2025Near-Optimal Second-Order Guarantees for Model-Based Adversarial Imitation Learningcited by 2
  10. 2021Multi-Agent Reinforcement Learning via Adaptive Kalman Temporal Difference and Successor Representationcited by 1
  11. 2022Online No-regret Model-Based Meta RL for Personalized Navigationcited by 0
  12. 2023Exploration in Model-based Reinforcement Learning with Randomized Rewardcited by 0
2 more
  1. 2024Fully-fused Multi-Layer Perceptrons on Intel Data Center GPUscited by 0
  2. 2025Digital Twin Calibration with Model-Based Reinforcement Learningcited by 0