mlsanityData to 7 Oct 2026

Generative Adversarial Imitation Learning with Neural Networks: Global Optimality and Convergence Rate

arXiv 2003.03709 ↗First posted 8 Mar 2020cs.LG math.OC stat.MLcited by 7

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Generative adversarial imitation learning (GAIL) demonstrates tremendous success in practice, especially when combined with neural networks. Different from reinforcement learning, GAIL learns both policy and reward function from expert (human) demonstration. Despite its empirical success, it remains unclear whether GAIL with neural networks converges to the globally optimal solution. The major difficulty comes from the nonconvex-nonconcave minimax optimization structure. To bridge the gap between practice and theory, we analyze a gradient-based algorithm with alternating updates and establish its sublinear convergence to the globally optimal solution. To the best of our knowledge, our analysis establishes the global optimality and convergence rate of GAIL with neural networks for the first time.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 19 of its 55 references that are in the map. First posted on arXiv 8 Mar 2020.

Builds on

19 of 55 references.

Closest ancestors · 2

Same direction, newest first

  1. 2020On Computation and Generalization of Generative Adversarial Imitation Learning9 Jan
  2. 2019On the Global Convergence of Imitation Learning: A Case for Linear Quadratic Regulator11 Jan

Other references · 17

  1. 2016Generative Adversarial Imitation Learningcited by 1,576
  2. 2017Wasserstein GANcited by 635
  3. 2019Provably Efficient Reinforcement Learning with Linear Function Approximationcited by 514
  4. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 400
  5. 2016SeqGAN: Sequence Generative Adversarial Nets with Policy Gradientcited by 375
  6. 2019Provably Efficient Exploration in Policy Optimizationcited by 203
  7. 2019Reinforcement Learning in Feature Space: Matrix Bandit, Kernels, and Regret Boundcited by 199
  8. 2019On the Theory of Policy Gradient Methods: Optimality, Approximation, and Distribution Shiftcited by 174
  9. 2019Neural Policy Gradient Methods: Global Optimality and Rates of Convergencecited by 157
  10. 2019Global Convergence of Policy Gradient Methods to (Almost) Locally Optimal Policiescited by 117
  11. 2017Imitating Driver Behavior with Generative Adversarial Networkscited by 92
  12. 2017Learning human behaviors from motion capture by adversarial imitationcited by 75
5 more
  1. 2019An Improved Convergence Analysis of Stochastic Variance-Reduced Policy Gradientcited by 58
  2. 2019Sample Efficient Policy Gradient Methods with Recursive Variance Reductioncited by 58
  3. 2017Socially Compliant Navigation through Raw Depth Inputs with Generative Adversarial Imitation Learningcited by 50
  4. 2019Neural Proximal/Trust Region Policy Optimization Attains Globally Optimal Policycited by 42
  5. 2019Policy Optimization Provably Converges to Nash Equilibria in Zero-Sum Linear Quadratic Gamescited by 38

Built on by

Cited by year

Citing papers per year: 2020: 1; 2021: 3; 2022: 2; 2023: 0; 2024: 0; 2025: 1.

7 papers.

  1. 2021Investigating Bi-Level Optimization for Learning and Vision from a Unified Perspective: A Survey and Beyondcited by 151
  2. 2021Online Apprenticeship Learningcited by 28
  3. 2020When Will Generative Adversarial Imitation Learning Algorithms Attain Global Convergencecited by 12
  4. 2021Efficient Performance Bounds for Primal-Dual Reinforcement Learning from Demonstrationscited by 9
  5. 2022Goal-Aware Generative Adversarial Imitation Learning from Imperfect Demonstration for Robotic Cloth Manipulationcited by 7
  6. 2022Model-Based Imitation Learning Using Entropy Regularization of Model and Policycited by 1
  7. 2025Understanding Inverse Reinforcement Learning under Overparameterization: Non-Asymptotic Analysis and Global Optimalitycited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2023Exploring Gradient Explosion in Generative Adversarial Imitation Learning: A Probabilistic Perspective
  2. 2024C-GAIL: Stabilizing Generative Adversarial Imitation Learning with Control Theory
  3. 2018Sample-Efficient Imitation Learning via Generative Adversarial Nets
  4. 2021Provably Efficient Generative Adversarial Imitation Learning for Online and Offline Setting with Linear Function Approximation
  5. 2020f-GAIL: Learning f-Divergence for Generative Adversarial Imitation Learning
  6. 2018Risk-Sensitive Generative Adversarial Imitation Learning
  7. 2026Adversarial Imitation Learning with General Function Approximation: Theoretical Analysis and Practical Algorithms
  8. 2021A Pragmatic Look at Deep Imitation Learning