mlsanityData to 7 Oct 2026

When Will Generative Adversarial Imitation Learning Algorithms Attain Global Convergence

arXiv 2006.13506 ↗First posted 24 Jun 2020cs.LG stat.MLcited by 12

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Generative adversarial imitation learning (GAIL) is a popular inverse reinforcement learning approach for jointly optimizing policy and reward from expert trajectories. A primary question about GAIL is whether applying a certain policy gradient algorithm to GAIL attains a global minimizer (i.e., yields the expert policy), for which existing understanding is very limited. Such global convergence has been shown only for the linear (or linear-type) MDP and linear (or linearizable) reward. In this paper, we study GAIL under general MDP and for nonlinear reward function classes (as long as the objective function is strongly concave with respect to the reward parameter). We characterize the global convergence with a sublinear rate for a broad range of commonly used policy gradient algorithms, all of which are implemented in an alternating manner with stochastic gradient ascent for reward update, including projected policy gradient (PPG)-GAIL, Frank-Wolfe policy gradient (FWPG)-GAIL, trust region policy optimization (TRPO)-GAIL and natural policy gradient (NPG)-GAIL. This is the first systematic theoretical study of GAIL for global convergence.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 20 of its 35 references that are in the map. First posted on arXiv 24 Jun 2020.

Builds on

20 of 35 references.

Closest ancestors · 6

Same direction, newest first

  1. 2020Generative Adversarial Imitation Learning with Neural Networks: Global Optimality and Convergence Rate8 Mar
  2. 2020On Computation and Generalization of Generative Adversarial Imitation Learning9 Jan
  3. 2019A Divergence Minimization Perspective on Imitation Learning Methods6 Nov
  4. 2019Imitation Learning as f-Divergence Minimization30 May
  5. 2019On the Global Convergence of Imitation Learning: A Case for Linear Quadratic Regulator11 Jan
  6. 2018An Algorithmic Perspective on Imitation Learning16 Nov

Other references · 14

  1. 2016Generative Adversarial Imitation Learningcited by 1,576
  2. 2016f-GAN: Training Generative Neural Samplers using Variational Divergence Minimizationcited by 504
  3. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 485
  4. 2019On the Theory of Policy Gradient Methods: Optimality, Approximation, and Distribution Shiftcited by 174
  5. 2016A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Modelscited by 167
  6. 2019Neural Policy Gradient Methods: Global Optimality and Rates of Convergencecited by 157
  7. 2019Solving a Class of Non-Convex Min-Max Games Using Iterative First Order Methodscited by 148
  8. 2020Near-Optimal Algorithms for Minimax Optimizationcited by 119
  9. 2019Global Convergence of Policy Gradient Methods to (Almost) Locally Optimal Policiescited by 117
  10. 2019Adaptive Trust Region Policy Optimization: Global Convergence and Faster Rates for Regularized MDPscited by 113
  11. 2019On the Sample Complexity of Actor-Critic Method for Reinforcement Learning with Function Approximationcited by 54
  12. 2019Neural Proximal/Trust Region Policy Optimization Attains Globally Optimal Policycited by 42
2 more
  1. 2020Reanalysis of Variance Reduced Temporal Difference Learningcited by 23
  2. 2020Improving Sample Complexity Bounds for (Natural) Actor-Critic Algorithmscited by 14

Built on by

Cited by year

Citing papers per year: 2022: 5; 2023: 3; 2024: 3; 2025: 1.

12 papers.

  1. 2022Maximum-Likelihood Inverse Reinforcement Learning with Finite-Time Guaranteescited by 42
  2. 2023When Demonstrations Meet Generative World Models: A Maximum Likelihood Framework for Offline Inverse Reinforcement Learningcited by 26
  3. 2022Structural Estimation of Markov Decision Processes in High-Dimensional State Space with Finite-Time Guaranteescited by 12
  4. 2024OLLIE: Imitation Learning from Offline Pretraining to Online Finetuningcited by 6
  5. 2022DM2: Decentralized Multi-Agent Reinforcement Learning for Distribution Matchingcited by 5
  6. 2023Exploring Gradient Explosion in Generative Adversarial Imitation Learning: A Probabilistic Perspectivecited by 4
  7. 2024C-GAIL: Stabilizing Generative Adversarial Imitation Learning with Control Theorycited by 3
  8. 2022Model-Based Imitation Learning Using Entropy Regularization of Model and Policycited by 1
  9. 2022Auto-Encoding Adversarial Imitation Learningcited by 1
  10. 2024Provably Efficient Off-Policy Adversarial Imitation Learning with Convergence Guaranteescited by 1
  11. 2025Understanding Inverse Reinforcement Learning under Overparameterization: Non-Asymptotic Analysis and Global Optimalitycited by 1
  12. 2023A Novel Variational Lower Bound for Inverse Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2020f-GAIL: Learning f-Divergence for Generative Adversarial Imitation Learning
  2. 2021A Pragmatic Look at Deep Imitation Learning
  3. 2019Situated GAIL: Multitask imitation using task-conditioned adversarial inverse reinforcement learning
  4. 2021Provably Efficient Generative Adversarial Imitation Learning for Online and Offline Setting with Linear Function Approximation
  5. 2024Diffusion-Reward Adversarial Imitation Learning
  6. 2026Adversarial Imitation Learning with General Function Approximation: Theoretical Analysis and Practical Algorithms
  7. 2018Risk-Sensitive Generative Adversarial Imitation Learning
  8. 2018Sample-Efficient Imitation Learning via Generative Adversarial Nets