mlsanityData to 7 Oct 2026

Intrinsic Reward Driven Imitation Learning via Generative Model

arXiv 2006.15061 ↗First posted 26 Jun 2020cs.LG cs.AI stat.MLcited by 35

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Imitation learning in a high-dimensional environment is challenging. Most inverse reinforcement learning (IRL) methods fail to outperform the demonstrator in such a high-dimensional environment, e.g., Atari domain. To address this challenge, we propose a novel reward learning module to generate intrinsic reward signals via a generative model. Our generative method can perform better forward state transition and backward action encoding, which improves the module's dynamics modeling ability in the environment. Thus, our module provides the imitation agent both the intrinsic intention of the demonstrator and a better exploration ability, which is critical for the agent to outperform the demonstrator. Empirical results show that our method outperforms state-of-the-art IRL methods on multiple Atari games, even with one-life demonstration. Remarkably, our method achieves performance that is up to 5 times the performance of the demonstration.

Key facts. Cited by 35 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 23 of its 42 references that are in the map. First posted on arXiv 26 Jun 2020.

Builds on

23 of 42 references.

Closest ancestors · 3

Same direction, newest first

  1. 2019Better-than-Demonstrator Imitation Learning via Automatically-Ranked Demonstrations9 Jul
  2. 2019Generative predecessor models for sample-efficient imitation learning1 Apr
  3. 2018Variational Discriminator Bottleneck: Improving Imitation Learning, Inverse RL, and GANs by Constraining Information Flow1 Oct

Other references · 20

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  4. 2013Auto-Encoding Variational Bayescited by 3,980
  5. 2017Deep reinforcement learning from human preferencescited by 3,846
  6. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  7. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  8. 2016OpenAI Gymcited by 1,684
  9. 2016Generative Adversarial Imitation Learningcited by 1,576
  10. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
  11. 2016Deep Variational Information Bottleneckcited by 915
  12. 2015Deep Learning and the Information Bottleneck Principlecited by 793
8 more
  1. 2017Masked Autoregressive Flow for Density Estimationcited by 680
  2. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 485
  3. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 400
  4. 2018Reward learning from human preferences and demonstrations in Ataricited by 314
  5. 2018Large-Scale Study of Curiosity-Driven Learningcited by 277
  6. 2019Extrapolating Beyond Suboptimal Demonstrations via Inverse Reinforcement Learning from Observationscited by 256
  7. 2018Adversarial Imitation via Variational Inverse Reinforcement Learningcited by 36
  8. 2018Inverse reinforcement learning for video gamescited by 26

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 6; 2022: 8; 2023: 5; 2024: 10; 2025: 5; 2026: 1.

35 papers.

Sort:

By area:

  1. 2021A Survey of Deep RL and IL for Autonomous Driving Policy Learningcited by 59
  2. 2021Imitation Learning: Progress, Taxonomies and Challengescited by 56
  3. 2024Deep Generative Models for Offline Policy Learning: Tutorial, Survey, and Perspectives on Future Directionscited by 20
  4. 2022R'enyi State Entropy for Exploration Acceleration in Reinforcement Learningcited by 18
  5. 2024A Survey of Imitation Learning Methods, Environments and Metricscited by 16
  6. 2021Imitation by Predicting Observationscited by 13
  7. 2022Understanding Adversarial Imitation Learning in Small Sample Regime: A Stage-coupled Analysiscited by 13
  8. 2023Automatic Intrinsic Reward Shaping for Exploration in Deep Reinforcement Learningcited by 11
  9. 2023CLUE: Calibrated Latent Guidance for Offline Reinforcement Learningcited by 11
  10. 2024RLeXplore: Accelerating Research in Intrinsically-Motivated Reinforcement Learningcited by 10
  11. 2023CLIP-Motion: Learning Reward Functions for Robotic Actions Using Consecutive Observationscited by 8
  12. 2024Pre-trained Visual Dynamics Representations for Efficient Policy Learningcited by 8
23 more
  1. 2026Olaf-World: Orienting Latent Actions for Video World Modelingcited by 8
  2. 2022Rewarding Episodic Visitation Discrepancy for Exploration in Reinforcement Learningcited by 7
  3. 2024Align Your Intents: Offline Imitation Learning via Optimal Transportcited by 7
  4. 2022From Psychological Curiosity to Artificial Curiosity: Curiosity-Driven Learning in Artificial Intelligence Taskscited by 6
  5. 2025IL-SOAR : Imitation Learning with Soft Optimistic Actor cRiticcited by 5
  6. 2022ProtoX: Explaining a Reinforcement Learning Agent via Prototypingcited by 4
  7. 2024Diversifying Policy Behaviors with Extrinsic Behavioral Curiositycited by 4
  8. 2024Imitation from Diverse Behaviors: Wasserstein Quality Diversity Imitation Learning with Single-Step Archive Explorationcited by 4
  9. 2025Generative Artificial Intelligence: Evolving Technology, Growing Societal Impact, and Opportunities for Information Systems Researchcited by 4
  10. 2023RLLTE: Long-Term Evolution Project of Reinforcement Learningcited by 3
  11. 2024Is Exploration All You Need? Effective Exploration Characteristics for Transfer in Reinforcement Learningcited by 3
  12. 2021Multimodal Reward Shaping for Efficient Exploration in Reinforcement Learningcited by 2
  13. 2022Intrinsically-Motivated Reinforcement Learning: A Brief Introductioncited by 2
  14. 2023Self-supervised network distillation: an effective approach to exploration in sparse reward environmentscited by 2
  15. 2024ELEMENT: Episodic and Lifelong Exploration via Maximum Entropycited by 2
  16. 2022A Survey on Deep Reinforcement Learning-based Approaches for Adaptation and Generalizationcited by 1
  17. 2022Tackling Visual Control via Multi-View Exploration Maximizationcited by 1
  18. 2025Beyond-Expert Performance with Limited Demonstrations: Efficient Imitation Learning with Double Explorationcited by 1
  19. 2025From Novelty to Imitation: Self-Distilled Rewards for Offline Reinforcement Learningcited by 1
  20. 2021Exploring Beyond-Demonstrator via Meta Learning-Based Reward Extrapolationcited by 0
  21. 2021Differential-Critic GAN: Generating What You Want by a Cue of Preferencescited by 0
  22. 2024External Model Motivated Agents: Reinforcement Learning for Enhanced Environment Samplingcited by 0
  23. 2025Hybrid-AIRL: Enhancing Inverse Reinforcement Learning with Supervised Expert Guidancecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022Towards Improving Exploration in Self-Imitation Learning using Intrinsic Motivation
  2. 2023Genetic Imitation Learning by Reward Extrapolation
  3. 2018Multi-Agent Generative Adversarial Imitation Learning
  4. 2019Random Expert Distillation: Imitation Learning via Expert Policy Support Estimation
  5. 2025Structured Imitation Learning of Interactive Policies through Inverse Games
  6. 2023Generating Personas for Games with Multimodal Adversarial Imitation Learning
  7. 2024Reward-free World Models for Online Imitation Learning
  8. 2019Deep Bayesian Reward Learning from Preferences