mlsanityData to 7 Oct 2026

Model-Based Active Exploration

arXiv 1810.12162 ↗First posted 29 Oct 2018cs.LG cs.AI cs.ITcited by 105

Not on the map. Why?

Abstract (arXiv)

Efficient exploration is an unsolved problem in Reinforcement Learning which is usually addressed by reactively rewarding the agent for fortuitously encountering novel situations. This paper introduces an efficient active exploration algorithm, Model-Based Active eXploration (MAX), which uses an ensemble of forward models to plan to observe novel events. This is carried out by optimizing agent behaviour with respect to a measure of novelty derived from the Bayesian perspective of exploration, which is estimated using the disagreement between the futures predicted by the ensemble members. We show empirically that in semi-random discrete environments where directed exploration is critical to make progress, MAX is at least an order of magnitude more efficient than strong baselines. MAX scales to high-dimensional continuous environments where it builds task-agnostic models that can be used for any downstream task.

Key facts. Cited by 105 papers in the map (data through 7 Oct 2026). First posted on arXiv 29 Oct 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 25; 2022: 20; 2023: 24; 2024: 15; 2025: 11; 2026: 10.

105 papers.

Sort:

By area:

  1. 2023Reward Model Ensembles Help Mitigate Overoptimizationcited by 225
  2. 2022Exploration in Deep Reinforcement Learning: A Surveycited by 158
  3. 2021Discovering and Achieving Goals via World Modelscited by 131
  4. 2023METRA: Scalable Unsupervised RL with Metric-Aware Abstractioncited by 80
  5. 2024Open-Endedness is Essential for Artificial Superhuman Intelligencecited by 79
  6. 2021A Survey of Exploration Methods in Reinforcement Learningcited by 68
  7. 2021Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domaincited by 57
  8. 2022Semantic Exploration from Language Abstractions and Pretrained Representationscited by 51
  9. 2022Optimizing Sequential Experimental Design with Deep Reinforcement Learningcited by 45
  10. 2021MADE: Exploration via Maximizing Deviation from Explored Regionscited by 40
  11. 2021Learning more skills through optimistic explorationcited by 37
  12. 2023Optimistic Active Exploration of Dynamical Systemscited by 32
38 more
  1. 2025Uncertainty-aware Latent Safety Filters for Avoiding Out-of-Distribution Failurescited by 30
  2. 2021Unsupervised Skill Discovery with Bottleneck Option Learningcited by 28
  3. 2023Planning Goals for Explorationcited by 28
  4. 2021How to Stay Curious while Avoiding Noisy TVs using Aleatoric Uncertainty Estimationcited by 27
  5. 2024ASID: Active Exploration for System Identification in Robotic Manipulationcited by 27
  6. 2022An information-theoretic perspective on intrinsic motivation in reinforcement learning: a surveycited by 25
  7. 2024Grounding Video Models to Actions through Goal Conditioned Explorationcited by 25
  8. 2022The Free Energy Principle for Perception and Action: A Deep Learning Perspectivecited by 24
  9. 2021Deep Reinforcement Learning Versus Evolution Strategies: A Comparative Surveycited by 23
  10. 2025Embodied AI: From LLMs to World Modelscited by 23
  11. 2021Curiosity-Driven Exploration via Latent Bayesian Surprisecited by 22
  12. 2023Grow Your Limits: Continuous Improvement with Real-World RL for Robotic Locomotioncited by 21
  13. 2024Mixture of Experts in a Mixture of RL settingscited by 21
  14. 2022A Differential Entropy Estimator for Training Neural Networkscited by 20
  15. 2023A Study of Global and Episodic Bonuses for Exploration in Contextual MDPscited by 19
  16. 2022A Bayesian Framework for Digital Twin-Based Control, Monitoring, and Data Collection in Wireless Systemscited by 18
  17. 2021Sample Efficient Reinforcement Learning via Model-Ensemble Exploration and Exploitationcited by 17
  18. 2021An Experimental Design Perspective on Model-Based Reinforcement Learningcited by 16
  19. 2023COPlanner: Plan to Roll Out Conservatively but to Explore Optimistically for Model-Based RLcited by 14
  20. 2023Reward-Free Curricula for Training Robust World Modelscited by 13
  21. 2024Prioritized Generative Replaycited by 13
  22. 2026World Action Verifier: Self-Improving World Models via Forward-Inverse Asymmetrycited by 13
  23. 2021Exploration and preference satisfaction trade-off in reward-free learningcited by 12
  24. 2021Backprop-Free Reinforcement Learning with Active Neural Generative Codingcited by 12
  25. 2022Learning General World Models in a Handful of Reward-Free Deploymentscited by 12
  26. 2021Active Learning of Abstract Plan Feasibilitycited by 11
  27. 2021Information is Power: Intrinsic Control via Information Capturecited by 11
  28. 2021Understanding the Origin of Information-Seeking Exploration in Probabilistic Objectives for Controlcited by 10
  29. 2022Generative Planning for Temporally Coordinated Exploration in Reinforcement Learningcited by 10
  30. 2022Exploring through Random Curiosity with General Value Functionscited by 10
  31. 2022Efficient Exploration in Resource-Restricted Reinforcement Learningcited by 10
  32. 2024R-AIF: Solving Sparse-Reward Robotic Tasks from Pixels with Active Inference and World Modelscited by 10
  33. 2024Online Intrinsic Rewards for Decision Making Agents from Large Language Model Feedbackcited by 10
  34. 2021Applications of the Free Energy Principle to Machine Learning and Neurosciencecited by 9
  35. 2022Active Exploration for Robotic Manipulationcited by 9
  36. 2021Imaginary Hindsight Experience Replay: Curious Model-based Learning for Sparse Reward Taskscited by 8
  37. 2022Exploration via Planning for Information about the Optimal Trajectorycited by 8
  38. 2025Posterior Behavioral Cloning: Pretraining BC Policies for Efficient RL Finetuningcited by 8

50 most cited of 105 All 105 (JSON)