mlsanityData to 14 Oct 2026

Imitation from Diverse Behaviors: Wasserstein Quality Diversity Imitation Learning with Single-Step Archive Exploration

arXiv 2411.06965 ↗First posted 11 Nov 2024cs.LG cs.AIcited by 4

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Learning diverse and high-performance behaviors from a limited set of demonstrations is a grand challenge. Traditional imitation learning methods usually fail in this task because most of them are designed to learn one specific behavior even with multiple demonstrations. Therefore, novel techniques for textitquality diversity imitation learning, which bridges the quality diversity optimization and imitation learning methods, are needed to solve the above challenge. This work introduces Wasserstein Quality Diversity Imitation Learning (WQDIL), which 1) improves the stability of imitation learning in the quality diversity setting with latent adversarial training based on a Wasserstein Auto-Encoder (WAE), and 2) mitigates a behavior-overfitting issue using a measure-conditioned reward function with a single-step archive exploration bonus. Empirically, our method significantly outperforms state-of-the-art IL methods, achieving near-expert or beyond-expert QD performance on the challenging continuous control tasks derived from MuJoCo environments.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 32 of its 52 references that are in the map. First posted on arXiv 11 Nov 2024.

Builds on

32 of 52 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024C-GAIL: Stabilizing Generative Adversarial Imitation Learning with Control Theory26 Feb
  2. 2023A Survey of Imitation Learning: Algorithms, Recent Developments, and Challenges5 Sep
  3. 2020Intrinsic Reward Driven Imitation Learning via Generative Model26 Jun
  4. 2020Primal Wasserstein Imitation Learning8 Jun

Other references · 28

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2017Improved Training of Wasserstein GANscited by 2,235
  3. 2016Generative Adversarial Imitation Learningcited by 1,597
  4. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,194
  5. 2016End to End Learning for Self-Driving Carscited by 1,009
  6. 2018Which Training Methods for GANs do actually Converge?cited by 679
  7. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 490
  8. 2015Illuminating search spaces by mapping elitescited by 425
  9. 2021Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulationcited by 407
  10. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 406
  11. 2017Wasserstein Auto-Encoderscited by 373
  12. 2016#Exploration: A Study of Count-Based Exploration for Deep Reinforcement Learningcited by 309
16 more
  1. 2018Large-Scale Study of Curiosity-Driven Learningcited by 280
  2. 2015Maximum Entropy Deep Inverse Reinforcement Learningcited by 176
  3. 2018Reinforcement and Imitation Learning for Diverse Visuomotor Skillscited by 126
  4. 2017Stabilizing Training of Generative Adversarial Networks through Regularizationcited by 118
  5. 2018Count-Based Exploration with the Successor Representationcited by 112
  6. 2021State Entropy Maximization with Random Encoders for Efficient Explorationcited by 108
  7. 2019Covariance Matrix Adaptation for the Rapid Illumination of Behavior Spacecited by 78
  8. 2021Differentiable Quality Diversitycited by 72
  9. 2017On the regularization of Wasserstein GANscited by 65
  10. 2020Quality-Diversity Optimization: a novel branch of stochastic optimizationcited by 60
  11. 2022Exploration via Elliptical Episodic Bonusescited by 51
  12. 2017From optimal transport to generative modeling: the VEGAN cookbookcited by 41
  13. 2022Approximating Gradients for Differentiable Quality Diversity in Reinforcement Learningcited by 38
  14. 2017Loss-Sensitive Generative Adversarial Networks on Lipschitz Densitiescited by 29
  15. 2022Covariance Matrix Adaptation MAP-Annealingcited by 25
  16. 2023Proximal Policy Gradient Arborescence for Quality Diversity Reinforcement Learningcited by 20

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 2.

4 papers.

  1. 2025FM-IRL: Flow-Matching for Reward Modeling and Policy Regularization in Reinforcement Learningcited by 3
  2. 2026Time-Annealed Perturbation Sampling: Diverse Generation for Diffusion Language Modelscited by 3
  3. 2025Soft Quality-Diversity Optimizationcited by 1
  4. 2026Adversarial Dual On-Policy Distillation from Expressive Teachercited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Latent Wasserstein Adversarial Imitation Learning
  2. 2024Towards Diverse Behaviors: A Benchmark for Imitation Learning with Human Demonstrations
  3. 2025Offline Learning of Controllable Diverse Behaviors
  4. 2020Wasserstein Distance guided Adversarial Imitation Learning with Reward Shape Exploration
  5. 2024Diversifying Policy Behaviors with Extrinsic Behavioral Curiosity
  6. 2024How to Leverage Diverse Demonstrations in Offline Imitation Learning
  7. 2019Wasserstein Adversarial Imitation Learning
  8. 2025Value from Observations: Towards Large-Scale Imitation Learning via Self-Improvement