mlsanityData to 7 Oct 2026

Ready Policy One: World Building Through Active Learning

arXiv 2002.02693 ↗First posted 7 Feb 2020cs.LG stat.MLcited by 33

Not on the map. Why?

Abstract (arXiv)

Model-Based Reinforcement Learning (MBRL) offers a promising direction for sample efficient learning, often achieving state of the art results for continuous control tasks. However, many existing MBRL methods rely on combining greedy policies with exploration heuristics, and even those which utilize principled exploration bonuses construct dual objectives in an ad hoc fashion. In this paper we introduce Ready Policy One (RP1), a framework that views MBRL as an active learning problem, where we aim to improve the world model in the fewest samples possible. RP1 achieves this by utilizing a hybrid objective function, which crucially adapts during optimization, allowing the algorithm to trade off reward v.s. exploration at different stages of learning. In addition, we introduce a principled mechanism to terminate sample collection once we have a rich enough trajectory batch to improve the model. We rigorously evaluate our method on a variety of continuous control tasks, and demonstrate statistically significant gains over existing approaches.

Key facts. Cited by 33 papers in the map (data through 7 Oct 2026). First posted on arXiv 7 Feb 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 15; 2022: 8; 2023: 5; 2024: 2; 2025: 1; 2026: 2.

33 papers.

Sort:

By area:

  1. 2021Discovering and Achieving Goals via World Modelscited by 131
  2. 2022Automated Reinforcement Learning (AutoRL): A Survey and Open Problemscited by 69
  3. 2021Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domaincited by 57
  4. 2021Tactical Optimism and Pessimism for Deep Reinforcement Learningcited by 55
  5. 2021Revisiting Design Choices in Offline Model-Based Reinforcement Learningcited by 50
  6. 2021Same State, Different Task: Continual Reinforcement Learning without Interferencecited by 42
  7. 2021Augmented World Models Facilitate Zero-Shot Dynamics Generalization From a Single Offline Environmentcited by 41
  8. 2022Adaptive Behavior Cloning Regularization for Stable Offline-to-Online Reinforcement Learningcited by 34
  9. 2022Stabilizing Off-Policy Deep Reinforcement Learning from Pixelscited by 32
  10. 2021Principled Exploration via Optimistic Bootstrapping and Backward Inductioncited by 26
  11. 2021Active Offline Policy Selectioncited by 26
  12. 2022Bayesian Generational Population-Based Trainingcited by 20
21 more
  1. 2021An Experimental Design Perspective on Model-Based Reinforcement Learningcited by 16
  2. 2023COPlanner: Plan to Roll Out Conservatively but to Explore Optimistically for Model-Based RLcited by 14
  3. 2023Reward-Free Curricula for Training Robust World Modelscited by 13
  4. 2021Exploration and preference satisfaction trade-off in reward-free learningcited by 12
  5. 2022Learning General World Models in a Handful of Reward-Free Deploymentscited by 12
  6. 2024Active Inference as a Model of Agencycited by 12
  7. 2021Centralized Model and Exploration Policy for Multi-Agent RLcited by 11
  8. 2022Exploration via Planning for Information about the Optimal Trajectorycited by 8
  9. 2023Predictable MDP Abstraction for Unsupervised Model-Based RLcited by 7
  10. 2021Model-Value Inconsistency as a Signal for Epistemic Uncertaintycited by 6
  11. 2023On the Theory of Risk-Aware Agents: Bridging Actor-Critic and Economicscited by 6
  12. 2021Look Before You Leap: Safe Model-Based Reinforcement Learning with Human Interventioncited by 5
  13. 2021Customs Fraud Detection in the Presence of Concept Driftcited by 4
  14. 2023Co-Learning Empirical Games and World Modelscited by 4
  15. 2021Bayesian Distributional Policy Gradientscited by 3
  16. 2022Active Exploration based on Information Gain by Particle Filter for Efficient Spatial Concept Formationcited by 2
  17. 2026Learning Task-Sufficient World Models by Synergizing Agentic Exploration and Structured Modelingcited by 2
  18. 2022Modelling non-reinforced preferences using selective attentioncited by 1
  19. 2024A Case for Validation Buffer in Pessimistic Actor-Criticcited by 0
  20. 2025Pruning Cannot Hurt Robustness: Certified Trade-offs in Reinforcement Learningcited by 0
  21. 2026PROWL: Prioritized Regret-Driven Optimization for World Model Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Enter the Void - Planning to Seek Entropy When Reward is Scarce
  2. 2020A Game Theoretic Framework for Model Based Reinforcement Learning
  3. 2024PWM: Policy Learning with Multi-Task World Models
  4. 2025Accelerating Model-Based Reinforcement Learning with State-Space World Models
  5. 2025Sample and Computationally Efficient Continuous-Time Reinforcement Learning with General Function Approximation
  6. 2026Theoretical Foundations and Effective Algorithms for Policy-Aware Simulator Learning
  7. 2022A Unified Framework for Alternating Offline Model Training and Policy Learning
  8. 2022A Survey on Model-based Reinforcement Learning