Data to 7 Oct 2026

Adversarial Imitation via Variational Inverse Reinforcement Learning

arXiv 1809.06404 ↗First posted 17 Sep 2018cs.LG cs.AI cs.ROcited by 36

On the map: Offline RL and imitation learning from data → Inverse reinforcement learning

Abstract (arXiv)

We consider a problem of learning the reward and policy from expert examples under unknown dynamics. Our proposed method builds on the framework of generative adversarial networks and introduces the empowerment-regularized maximum-entropy inverse reinforcement learning to learn near-optimal rewards and policies. Empowerment-based regularization prevents the policy from overfitting to expert demonstrations, which advantageously leads to more generalized behaviors that result in learning near-optimal rewards. Our method simultaneously learns empowerment through variational information maximization along with the reward and policy under the adversarial learning formulation. We evaluate our approach on various high-dimensional complex control tasks. We also test our learned rewards in challenging transfer learning problems where training and testing environments are made to be different from each other in terms of dynamics or structure. The results show that our proposed method not only learns near-optimal rewards and policies that are matching expert behavior but also performs significantly better than state-of-the-art inverse reinforcement learning algorithms.

Key facts. Cited by 36 papers in the map (data through 7 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 11 of its 28 references that are in the map. First posted on arXiv 17 Sep 2018.

Builds on

11 of 28 references.

References · 11

2017Proximal Policy Optimization Algorithmscited by 14,441
2015Trust Region Policy Optimizationcited by 2,503
2016Generative Adversarial Imitation Learningcited by 1,576
2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 485
2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 400
2016A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Modelscited by 167
2015Variational Information Maximisation for Intrinsically Motivated Reinforcement Learningcited by 142
2013Empowerment -- an Introductioncited by 65
2017Robot Autonomy for Surgerycited by 16
2018Intrinsically motivated reinforcement learning for human-robot interaction in the real-worldcited by 12
2017Show, Attend and Interact: Perceivable Human-Robot Social Interaction through Neural Attention Q-Networkcited by 2

Built on by

Cited by year

Citing papers per year: 2019: 5; 2020: 4; 2021: 8; 2022: 6; 2023: 8; 2024: 5.

36 papers.

Sort:

By area:

2019Extrapolating Beyond Suboptimal Demonstrations via Inverse Reinforcement Learning from Observationscited by 256
2019Imitation Learning as f-Divergence Minimizationcited by 121
2019State Alignment-based Imitation Learningcited by 73
2020f-IRL: Inverse Reinforcement Learning via State Marginal Matchingcited by 67
2020Strictly Batch Imitation Learning by Energy-based Distribution Matchingcited by 54
2020Intrinsic Reward Driven Imitation Learning via Generative Modelcited by 35
2021Learning from Imperfect Demonstrations from Agents with Varying Dynamicscited by 26
2024SINDy-RL: Interpretable and Efficient Model-Based Reinforcement Learningcited by 24
2019Imitation Learning from Video by Leveraging Proprioceptioncited by 23
2023CLARE: Conservative Model-Based Reward Learning for Offline Inverse Reinforcement Learningcited by 20
2021Offline Inverse Reinforcement Learningcited by 14
2021Recent Advances in Leveraging Human Guidance for Sequential Decision-Making Taskscited by 13

24 more

2021Inverse Contextual Bandits: Learning How Behavior Evolves over Timecited by 12
2021Learning Feasibility to Imitate Demonstrators with Different Dynamicscited by 10
2022Receding Horizon Inverse Reinforcement Learningcited by 10
2023Inverse Decision Modeling: Learning Interpretable Representations of Behaviorcited by 10
2023Time-series Generation by Contrastive Imitationcited by 8
2022Goal-Aware Generative Adversarial Imitation Learning from Imperfect Demonstration for Robotic Cloth Manipulationcited by 7
2020Efficiently Guiding Imitation Learning Agents with Human Gazecited by 6
2022Learning Multi-Task Transferable Rewards via Variational Inverse Reinforcement Learningcited by 5
2023Learning Reward for Physical Skills using Large Language Modelcited by 5
2023Is Inverse Reinforcement Learning Harder than Standard Reinforcement Learning? A Theoretical Perspectivecited by 4
2019Hindsight Generative Adversarial Imitation Learningcited by 3
2021Off-Dynamics Inverse Reinforcement Learning from Hetero-Domaincited by 3
2023Explaining by Imitating: Understanding Decisions by Interpretable Policy Learningcited by 3
2021A Generalised Inverse Reinforcement Learning Frameworkcited by 2
2022Towards Interpretable Deep Reinforcement Learning Models via Inverse Reinforcement Learningcited by 2
2022Auto-Encoding Adversarial Imitation Learningcited by 1
2024IBCB: Efficient Inverse Batched Contextual Bandit for Behavioral Evolution Historycited by 1
2024SuPLE: Robot Learning with Lyapunov Rewardscited by 1
2021Extracting Expert's Goals by What-if Interpretable Modelingcited by 0
2022Context-Hierarchy Inverse Reinforcement Learningcited by 0
2023Sample-efficient Adversarial Imitation Learningcited by 0
2023A Novel Variational Lower Bound for Inverse Reinforcement Learningcited by 0
2024Rethinking Adversarial Inverse Reinforcement Learning: Policy Imitation, Transferable Reward Recovery and Algebraic Equilibrium Proofcited by 0
2024On Reward Transferability in Adversarial Inverse Reinforcement Learning: Insights from Random Matrix Theorycited by 0

Similar, not cited

Close in wording, no citation either way