mlsanityData to 14 Oct 2026

Coupled Distributional Random Expert Distillation for World Model Online Imitation Learning

arXiv 2505.02228 ↗First posted 4 May 2025cs.LG cs.AIcited by 0

On the map: World models → Embodied and robot world models

also in 1 more
Abstract (arXiv)

Imitation Learning (IL) has achieved remarkable success across various domains, including robotics, autonomous driving, and healthcare, by enabling agents to learn complex behaviors from expert demonstrations. However, existing IL methods often face instability challenges, particularly when relying on adversarial reward or value formulations in world model frameworks. In this work, we propose a novel approach to online imitation learning that addresses these limitations through a reward model based on random network distillation (RND) for density estimation. Our reward model is built on the joint estimation of expert and behavioral distributions within the latent space of the world model. We evaluate our method across diverse benchmarks, including DMControl, Meta-World, and ManiSkill2, showcasing its ability to deliver stable performance and achieve expert-level results in both locomotion and manipulation tasks. Our approach demonstrates improved stability over adversarial methods while maintaining expert-level performance.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Embodied and robot world models, one of 11 directions in World models. Builds on 50 of its 55 references that are in the map. First posted on arXiv 4 May 2025.

Builds on

50 of 55 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024Reward-free World Models for Online Imitation Learning17 Oct
  2. 2024Efficient Imitation Learning with Conservative World Models21 May
  3. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusion7 Mar
  4. 2023DITTO: Offline Imitation Learning with World Models6 Feb

Other references · 46

  1. 2016Layer Normalizationcited by 5,669
  2. 2017Improved Training of Wasserstein GANscited by 2,235
  3. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
  4. 2016Generative Adversarial Imitation Learningcited by 1,597
  5. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,353
  6. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
  7. 2023Mastering Diverse Domains through World Modelscited by 1,253
  8. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
  9. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,204
  10. 2020Mastering Atari with Discrete World Modelscited by 1,007
  11. 2018Exploration by Random Network Distillationcited by 993
  12. 2018DeepMind Control Suitecited by 956
34 more
  1. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 773
  2. 2020MOPO: Model-based Offline Policy Optimizationcited by 636
  3. 2023TD-MPC2: Scalable, Robust World Models for Continuous Controlcited by 512
  4. 2021Implicit Behavioral Cloningcited by 452
  5. 2022Temporal Difference Learning for Model Predictive Controlcited by 397
  6. 2023ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skillscited by 327
  7. 2021Mastering Atari Games with Limited Datacited by 237
  8. 2021IQ-Learn: Inverse soft-Q Learning for Imitationcited by 193
  9. 2019Imitation Learning via Off-Policy Distribution Matchingcited by 183
  10. 2018Model-Based Value Estimation for Efficient Model-Free Reinforcement Learningcited by 156
  11. 2022Model-Based Imitation Learning for Urban Drivingcited by 150
  12. 2024Stop Regressing: Training Value Functions via Classification for Scalable Deep RLcited by 144
  13. 2020Language-Conditioned Imitation Learning for Robot Manipulation Taskscited by 136
  14. 2022VIOLA: Imitation Learning for Vision-Based Manipulation with Object Proposal Priorscited by 136
  15. 2015Model Predictive Path Integral Control using Covariance Variable Importance Samplingcited by 124
  16. 2024PLUTO: Pushing the Limit of Imitation Learning-based Planning for Autonomous Drivingcited by 84
  17. 2023LOTUS: Continual Imitation Learning for Robot Manipulation Through Unsupervised Skill Discoverycited by 67
  18. 2022Symphony: Learning Realistic and Diverse Agents for Autonomous Driving Simulationcited by 61
  19. 2023Deep Imitation Learning for Humanoid Loco-manipulation through Human Teleoperationcited by 54
  20. 2020Model-Based Inverse Reinforcement Learning from Visual Demonstrationscited by 52
  21. 2024DiffuseLoco: Real-Time Legged Locomotion Control with Diffusion from Offline Datasetscited by 49
  22. 2022Hierarchical Model-Based Imitation Learning for Planning in Autonomous Drivingcited by 44
  23. 2019Random Expert Distillation: Imitation Learning via Expert Policy Support Estimationcited by 42
  24. 2021Visual Adversarial Imitation Learning using Variational Modelscited by 42
  25. 2024EfficientZero V2: Mastering Discrete and Continuous Control with Limited Datacited by 40
  26. 2024Exploration and Anti-Exploration with Distributional Random Network Distillationcited by 33
  27. 2024Hybrid Inverse Reinforcement Learningcited by 32
  28. 2016Model-based Adversarial Imitation Learningcited by 29
  29. 2022Planning for Sample Efficient Imitation Learningcited by 20
  30. 2022Discriminator-Guided Model-Based Offline Imitation Learningcited by 17
  31. 2023A Coupled Flow Approach to Imitation Learningcited by 12
  32. 2024Learning Speed-Adaptive Walking Agent Using Imitation Learning with Physics-Informed Simulationcited by 5
  33. 2023Contextualized Policy Recovery: Modeling and Interpreting Medical Decisions with Adaptive Imitation Learningcited by 4
  34. 2020Support-weighted Adversarial Imitation Learningcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Efficient Active Imitation Learning with Random Network Distillation
  2. 2025Near-Optimal Second-Order Guarantees for Model-Based Adversarial Imitation Learning
  3. 2026Provable Benefits of Regularization: Fast Rates for Adversarial Imitation Learning
  4. 2026Provably Efficient Policy-Reward Co-Pretraining for Adversarial Imitation Learning
  5. 2026Learning Diverse Skills for Behavior Models with Mixture of Experts
  6. 2023Provably Efficient Adversarial Imitation Learning with Unknown Transitions
  7. 2024Diffusion-Reward Adversarial Imitation Learning
  8. 2025SD2AIL: Adversarial Imitation Learning from Synthetic Demonstrations via Diffusion Models