mlsanityData to 14 Oct 2026

Random Expert Distillation: Imitation Learning via Expert Policy Support Estimation

arXiv 1905.06750 ↗First posted 16 May 2019cs.LG stat.MLcited by 42

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

We consider the problem of imitation learning from a finite set of expert trajectories, without access to reinforcement signals. The classical approach of extracting the expert's reward function via inverse reinforcement learning, followed by reinforcement learning is indirect and may be computationally expensive. Recent generative adversarial methods based on matching the policy distribution between the expert and the agent could be unstable during training. We propose a new framework for imitation learning by estimating the support of the expert policy to compute a fixed reward function, which allows us to re-frame imitation learning within the standard reinforcement learning setting. We demonstrate the efficacy of our reward function on both discrete and continuous domains, achieving comparable or better performance than the state of the art under different reinforcement learning algorithms.

Key facts. Cited by 42 papers in the map (data through 14 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 14 of its 31 references that are in the map. First posted on arXiv 16 May 2019.

Builds on

14 of 31 references.

References · 14

  1. 2016Improved Techniques for Training GANscited by 2,970
  2. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,626
  3. 2015Trust Region Policy Optimizationcited by 2,580
  4. 2016Generative Adversarial Imitation Learningcited by 1,597
  5. 2016End to End Learning for Self-Driving Carscited by 1,009
  6. 2018Exploration by Random Network Distillationcited by 993
  7. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 715
  8. 2017Towards Principled Methods for Training Generative Adversarial Networkscited by 464
  9. 2018ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worstcited by 397
  10. 2017Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuningcited by 297
  11. 2017Are GANs Created Equal? A Large-Scale Studycited by 267
  12. 2016A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Modelscited by 168
2 more
  1. 2015Learning Continuous Control Policies by Stochastic Value Gradientscited by 162
  2. 2017Deeply AggreVaTeD: Differentiable Imitation Learning for Sequential Predictioncited by 93

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2019: 1; 2020: 7; 2021: 11; 2022: 6; 2023: 8; 2024: 3; 2025: 5; 2026: 1.

42 papers.

Sort:

By area:

  1. 2023A Survey of Imitation Learning: Algorithms, Recent Developments, and Challengescited by 148
  2. 2020Primal Wasserstein Imitation Learningcited by 121
  3. 2020f-IRL: Inverse Reinforcement Learning via State Marginal Matchingcited by 67
  4. 2020Strictly Batch Imitation Learning by Energy-based Distribution Matchingcited by 54
  5. 2021Offline Reinforcement Learning as Anti-Explorationcited by 44
  6. 2022Maximum-Likelihood Inverse Reinforcement Learning with Finite-Time Guaranteescited by 42
  7. 2020Energy-Based Imitation Learningcited by 40
  8. 2021Core Challenges in Embodied Vision-Language Planningcited by 30
  9. 2019SQIL: Imitation Learning via Reinforcement Learning with Sparse Rewardscited by 28
  10. 2022Minimax Optimal Online Imitation Learning via Replay Estimationcited by 25
  11. 2021Continuous Control with Action Quantization from Demonstrationscited by 22
  12. 2023On The Fragility of Learned Reward Functionscited by 21
30 more
  1. 2023DITTO: Offline Imitation Learning with World Modelscited by 21
  2. 2023MAHALO: Unifying Offline Reinforcement Learning and Imitation Learning from Observationscited by 18
  3. 2021Hyperparameter Selection for Imitation Learningcited by 17
  4. 2021Imitation Learning by Reinforcement Learningcited by 17
  5. 2020Lipschitzness Is All You Need To Tame Off-policy Generative Adversarial Imitation Learningcited by 15
  6. 2021SoftDICE for Imitation Learning: Rethinking Off-policy Distribution Matchingcited by 14
  7. 2021Unifying Behavioral and Response Diversity for Open-ended Learning in Zero-sum Gamescited by 14
  8. 2020Universal Value Density Estimation for Imitation Learning and Goal-Conditioned Reinforcement Learningcited by 12
  9. 2022Imitation Learning from Observations under Transition Model Disparitycited by 11
  10. 2021A Pragmatic Look at Deep Imitation Learningcited by 9
  11. 2022Imitation Learning by State-Only Distribution Matchingcited by 6
  12. 2023A Strong Baseline for Batch Imitation Learningcited by 5
  13. 2021Seeing Differently, Acting Similarly: Heterogeneously Observable Imitation Learningcited by 4
  14. 2023Get Back Here: Robust Imitation by Return-to-Distribution Planningcited by 4
  15. 2024Robust Imitation Learning for Automated Game Testingcited by 4
  16. 2020Support-weighted Adversarial Imitation Learningcited by 3
  17. 2021Deterministic and Discriminative Imitation (D2-Imitation): Revisiting Adversarial Imitation for Sample Efficiencycited by 3
  18. 2024Expert Proximity as Surrogate Rewards for Single Demonstration Imitation Learningcited by 2
  19. 2022Auto-Encoding Adversarial Imitation Learningcited by 1
  20. 2024Robot See, Robot Do: Imitation Reward for Noisy Financial Environmentscited by 1
  21. 2025Generalizing Behavior via Inverse Reinforcement Learning with Closed-Form Reward Centroidscited by 1
  22. 2025Distributionally Robust Imitation Learning: Layered Control Architecture for Certifiable Autonomycited by 1
  23. 2026Learning from Demonstrations via Capability-Aware Goal Samplingcited by 1
  24. 2021Probability Density Estimation Based Imitation Learningcited by 0
  25. 2022GAIL-PT: A Generic Intelligent Penetration Testing Framework with Generative Adversarial Imitation Learningcited by 0
  26. 2023Core Challenges in Embodied Vision-Language Planningcited by 0
  27. 2023Human-Like Autonomous Driving on Dense Trafficcited by 0
  28. 2025Coupled Distributional Random Expert Distillation for World Model Online Imitation Learningcited by 0
  29. 2025Learning from Observation: A Survey of Recent Advancescited by 0
  30. 2025Noise-Guided Transport for Imitation Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Enabling Off-Policy Imitation Learning with Deep Actor Critic Stabilization
  2. 2019A New Framework for Query Efficient Active Imitation Learning
  3. 2024Inverse Delayed Reinforcement Learning
  4. 2019Reinforced Imitation in Heterogeneous Action Space
  5. 2022IL-flOw: Imitation Learning from Observation using Normalizing Flows
  6. 2020f-GAIL: Learning f-Divergence for Generative Adversarial Imitation Learning
  7. 2023Provably Efficient Adversarial Imitation Learning with Unknown Transitions
  8. 2018Multi-Agent Generative Adversarial Imitation Learning