mlsanityData to 7 Oct 2026

Beyond Conservatism: Recoverability-Conditioned Exploration for Model-Based Imitation Learning

arXiv 2609.33336 ↗First posted 27 Sep 2026cs.LG cs.AIcited by 0

On the map: World models → Latent dynamics models and model-based RL

also in 1 more
Abstract (arXiv)

Model-based imitation learning (MBIL) improves real-environment interaction efficiency by optimizing policies on imagined rollouts from a learned world model. However, the gap between model-induced and real-environment occupancies makes policy learning sensitive to model error. Conservative MBIL mitigates model exploitation during policy optimization, but when real-environment interactions are collected by the same conservative policy, uncertain regions around the expert distribution remain insufficiently sampled. Generic uncertainty-driven exploration, on the other hand, may allocate interaction to novel but task-irrelevant dynamics. We propose REcoverability-CONditioned Exploration for Model-Based Imitation Learning (RECON). RECON separates conservative policy learning from active data collection by maintaining a main policy for task execution and an explorer for real-environment interaction. The explorer is optimized based on epistemic uncertainty conditioned on recoverability estimated from multi-step main-policy imagination, focusing data collection on unknown states from which the main policy can still return toward expert behavior. Experiments on locomotion, navigation and manipulation show consistent gains in interaction efficiency, imitation performance, and robustness, indicating that RECON directs real-environment interaction toward recovery regions around the expert distribution that are underexplored by prior methods, and thereby learns a world model better suited for imitation.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Latent dynamics models and model-based RL, one of 11 directions in World models. Builds on 37 of its 42 references that are in the map. First posted on arXiv 27 Sep 2026.

Builds on

37 of 42 references.

Closest ancestors · 8

Same direction, newest first

  1. 2025Offline vs. Online Learning in Model-based RL: Lessons for Data Collection Strategies6 Sep
  2. 2023TD-MPC2: Scalable, Robust World Models for Continuous Control25 Oct
  3. 2023Mastering Diverse Domains through World Models10 Jan
  4. 2020Mastering Atari with Discrete World Models5 Oct
  5. 2020Planning to Explore via Self-Supervised World Models12 May
  6. 2019Dream to Control: Learning Behaviors by Latent Imagination3 Dec
  7. 2019When to Trust Your Model: Model-Based Policy Optimization19 Jun
  8. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models30 May

Other references · 29

  1. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  2. 2016Generative Adversarial Imitation Learningcited by 1,576
  3. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
  4. 2020MOPO: Model-based Offline Policy Optimizationcited by 630
  5. 2020MOReL : Model-Based Offline Reinforcement Learningcited by 582
  6. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 485
  7. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 400
  8. 2022Temporal Difference Learning for Model Predictive Controlcited by 375
  9. 2021COMBO: Conservative Offline Model-Based Policy Optimizationcited by 365
  10. 2019Challenges of Real-World Reinforcement Learningcited by 282
  11. 2018Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learningcited by 203
  12. 2017DART: Noise Injection for Robust Imitation Learningcited by 187
17 more
  1. 2022Model-Based Imitation Learning for Urban Drivingcited by 149
  2. 2018Model-Based Active Explorationcited by 105
  3. 2021MobILE: Model-Based Imitation Learning From Observation Alonecited by 44
  4. 2021Visual Adversarial Imitation Learning using Variational Modelscited by 42
  5. 2024JUICER: Data-Efficient Imitation Learning for Robotic Assemblycited by 36
  6. 2024Hybrid Inverse Reinforcement Learningcited by 31
  7. 2016Model-based Adversarial Imitation Learningcited by 29
  8. 2023DITTO: Offline Imitation Learning with World Modelscited by 20
  9. 2024Efficient Imitation Learning with Conservative World Modelscited by 11
  10. 2023TOM: Learning Policy-Aware Models for Model-Based Reinforcement Learning via Transition Occupancy Matchingcited by 10
  11. 2023SeMAIL: Eliminating Distractors in Visual Imitation via Separated Modelscited by 10
  12. 2024Offline Imitation Learning with Model-based Reverse Augmentationcited by 8
  13. 2022Robust Imitation of a Few Demonstrations with a Backwards Modelcited by 7
  14. 2023Offline Retraining for Online RL: Decoupled Policy Learning to Mitigate Exploration Biascited by 7
  15. 2024Reward-free World Models for Online Imitation Learningcited by 7
  16. 2023Action Inference by Maximising Evidence: Zero-Shot Imitation from Observation with World Modelscited by 5
  17. 2025Poke and Strike: Learning Task-Informed Exploration Policiescited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2021Learning from Guided Play: A Scheduled Hierarchical Approach for Improving Exploration in Adversarial Imitation Learning
  2. 2022Learning from Guided Play: Improving Exploration for Adversarial Imitation Learning with Simple Auxiliary Tasks
  3. 2025A Smooth Sea Never Made a Skilled SAILOR: Robust Imitation via Learning to Search
  4. 2022Robust Learning from Observation with Model Misspecification
  5. 2026Is Online Interaction Necessary for Recovery? A Minimalist Approach to Robust Planning via Perturbation ↗
  6. 2025Using Non-Expert Data to Robustify Imitation Learning via Offline Reinforcement Learning
  7. 2025RoboScape-R: Unified Reward-Observation World Models for Generalizable Robotics Training via RL
  8. 2023Provably Efficient Adversarial Imitation Learning with Unknown Transitions