mlsanityData to 14 Oct 2026

A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Models

arXiv 1611.03852 ↗First posted 11 Nov 2016cs.LG cs.AIcited by 168

Not on the map. Why?

Cited by 111 in Offline RL and imitation learning from data, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Generative adversarial networks (GANs) are a recently proposed class of generative models in which a generator is trained to optimize a cost function that is being simultaneously learned by a discriminator. While the idea of learning cost functions is relatively new to the field of generative modeling, learning costs has long been studied in control and reinforcement learning (RL) domains, typically for imitation learning from demonstrations. In these fields, learning cost function underlying observed behavior is known as inverse reinforcement learning (IRL) or inverse optimal control. While at first the connection between cost learning in RL and cost learning in generative modeling may appear to be a superficial one, we show in this paper that certain IRL methods are in fact mathematically equivalent to GANs. In particular, we demonstrate an equivalence between a sample-based algorithm for maximum entropy IRL and a GAN in which the generator's density can be evaluated and is provided as an additional input to the discriminator. Interestingly, maximum entropy IRL is a special case of an energy-based model. We discuss the interpretation of GANs as an algorithm for training energy-based models, and relate this interpretation to other recent work that seeks to connect GANs and EBMs. By formally highlighting the connection between GANs, IRL, and EBMs, we hope that researchers in all three communities can better identify and apply transferable ideas from one domain to another, particularly for developing more stable and scalable algorithms: a major challenge in all three domains.

Key facts. Cited by 168 papers in the map (data through 14 Oct 2026). First posted on arXiv 11 Nov 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2018: 8; 2019: 14; 2020: 17; 2021: 30; 2022: 23; 2023: 27; 2024: 21; 2025: 16; 2026: 12.

168 papers.

Sort:

By area:

  1. 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,114
  2. 2018An Algorithmic Perspective on Imitation Learningcited by 459
  3. 2021Offline Reinforcement Learning with Fisher Divergence Critic Regularizationcited by 238
  4. 2018Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learningcited by 204
  5. 2019A Divergence Minimization Perspective on Imitation Learning Methodscited by 190
  6. 2024Direct Nash Optimization: Teaching Language Models to Self-Improve with General Preferencescited by 178
  7. 2019Imitation Learning as f-Divergence Minimizationcited by 123
  8. 2021Generalized Decision Transformer for Offline Hindsight Information Matchingcited by 106
  9. 2018Variational Discriminator Bottleneck: Improving Imitation Learning, Inverse RL, and GANs by Constraining Information Flowcited by 105
  10. 2021Demonstration-Guided Reinforcement Learning with Learned Skillscited by 76
  11. 2020Safe Imitation Learning via Fast Bayesian Reward Inference from Preferencescited by 73
  12. 2019From Language to Goals: Inverse Reinforcement Learning for Vision-Based Instruction Followingcited by 67
38 more
  1. 2020f-IRL: Inverse Reinforcement Learning via State Marginal Matchingcited by 67
  2. 2021A Survey of Deep RL and IL for Autonomous Driving Policy Learningcited by 61
  3. 2019Multi-Agent Adversarial Inverse Reinforcement Learningcited by 60
  4. 2021Trajectory Prediction with Latent Belief Energy-Based Modelcited by 60
  5. 2021Identifiability in inverse reinforcement learningcited by 55
  6. 2024A Dense Reward View on Aligning Text-to-Image Diffusion with Preferencecited by 55
  7. 2020Strictly Batch Imitation Learning by Energy-based Distribution Matchingcited by 54
  8. 2019Meta-Inverse Reinforcement Learning with Probabilistic Context Variablescited by 50
  9. 2019Task-Relevant Adversarial Imitation Learningcited by 45
  10. 2021Confidence-Aware Imitation Learning from Demonstrations with Varying Optimalitycited by 44
  11. 2019Random Expert Distillation: Imitation Learning via Expert Policy Support Estimationcited by 42
  12. 2021Visual Adversarial Imitation Learning using Variational Modelscited by 42
  13. 2022Maximum-Likelihood Inverse Reinforcement Learning with Finite-Time Guaranteescited by 42
  14. 2020Energy-Based Imitation Learningcited by 40
  15. 2023PARL: A Unified Framework for Policy Alignment in Reinforcement Learning from Human Feedbackcited by 40
  16. 2018Sample-Efficient Imitation Learning via Generative Adversarial Netscited by 36
  17. 2018Adversarial Imitation via Variational Inverse Reinforcement Learningcited by 36
  18. 2019SQIL: Imitation Learning via Reinforcement Learning with Sparse Rewardscited by 28
  19. 2018Inverse reinforcement learning for video gamescited by 26
  20. 2025Reward Models in Deep Reinforcement Learning: A Surveycited by 25
  21. 2021PsiPhi-Learning: Reinforcement Learning with Demonstrations using Successor Features and Inverse Temporal Difference Learningcited by 23
  22. 2020Adversarial Soft Advantage Fitting: Imitation Learning without Policy Optimizationcited by 22
  23. 2024Real-World Offline Reinforcement Learning from Vision Language Model Feedbackcited by 22
  24. 2018Dialogue Generation: From Imitation Learning to Inverse Reinforcement Learningcited by 21
  25. 2020Sequential Anomaly Detection using Inverse Reinforcement Learningcited by 20
  26. 2020f-GAIL: Learning f-Divergence for Generative Adversarial Imitation Learningcited by 20
  27. 2021Domain-Robust Visual Imitation Learning with Mutual Information Constraintscited by 20
  28. 2022Emergent Bartering Behaviour in Multi-Agent Reinforcement Learningcited by 19
  29. 2023Fantastic Rewards and How to Tame Them: A Case Study on Reward Learning for Task-oriented Dialogue Systemscited by 19
  30. 2024Vision-Language Navigation with Energy-Based Policycited by 19
  31. 2020Non-Adversarial Imitation Learning and its Connections to Adversarial Methodscited by 17
  32. 2023Constrained Policy Optimization with Explicit Behavior Density for Offline Reinforcement Learningcited by 16
  33. 2023EGC: Image Generation and Classification via a Diffusion Energy-Based Modelcited by 16
  34. 2024Diffusing States and Matching Scores: A New Framework for Imitation Learningcited by 16
  35. 2023A survey and taxonomy of loss functions in machine learningcited by 15
  36. 2023Invariant Causal Imitation Learning for Generalizable Policiescited by 15
  37. 2025Generative Models in Decision Making: A Surveycited by 15
  38. 2025AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learningcited by 14

50 most cited of 168 All 168 (JSON)