mlsanityData to 7 Oct 2026

On the Global Convergence of Imitation Learning: A Case for Linear Quadratic Regulator

arXiv 1901.03674 ↗First posted 11 Jan 2019cs.LG cs.AI math.OCcited by 20

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

We study the global convergence of generative adversarial imitation learning for linear quadratic regulators, which is posed as minimax optimization. To address the challenges arising from non-convex-concave geometry, we analyze the alternating gradient algorithm and establish its Q-linear rate of convergence to a unique saddle point, which simultaneously recovers the globally optimal policy and reward function. We hope our results may serve as a small step towards understanding and taming the instability in imitation learning as well as in more general non-convex-concave alternating minimax optimization that arises from reinforcement learning and generative adversarial learning.

Key facts. Cited by 20 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 27 of its 48 references that are in the map. First posted on arXiv 11 Jan 2019.

Builds on

27 of 48 references.

References · 27

  1. 2016Generative Adversarial Imitation Learningcited by 1,576
  2. 2014Convex Optimization: Algorithms and Complexitycited by 569
  3. 2017Evolution Strategies as a Scalable Alternative to Reinforcement Learningcited by 555
  4. 2015Escaping From Saddle Points --- Online Stochastic Gradient for Tensor Decompositioncited by 241
  5. 2018Global Convergence of Policy Gradient Methods for the Linear Quadratic Regulatorcited by 211
  6. 2018Learning Without Mixing: Towards A Sharp Analysis of Linear System Identificationcited by 158
  7. 2017On the Sample Complexity of the Linear Quadratic Regulatorcited by 151
  8. 2018A Tour of Reinforcement Learning: The View from Continuous Controlcited by 129
  9. 2017No Spurious Local Minima in Nonconvex Low Rank Problems: A Unified Geometric Analysiscited by 127
  10. 2018On the Power of Over-parametrization in Neural Networks with Quadratic Activationcited by 102
  11. 2016A Geometric Analysis of Phase Retrievalcited by 92
  12. 2018Regret Bounds for Robust Adaptive Control of the Linear Quadratic Regulatorcited by 86
15 more
  1. 2016Global Optimality of Local Search for Low Rank Matrix Recoverycited by 81
  2. 2017Learning One-hidden-layer Neural Networks with Landscape Designcited by 79
  3. 2016Gradient Descent Learns Linear Dynamical Systemscited by 78
  4. 2018The Gap Between Model-Based and Model-Free Methods on the Linear Quadratic Regulator: An Asymptotic Viewpointcited by 36
  5. 2018Safely Learning to Control the Constrained Linear Quadratic Regulatorcited by 31
  6. 2018Linear Convergence of the Primal-Dual Gradient Method for Convex-Concave Saddle Point Problems without Strong Convexitycited by 28
  7. 2017Least-Squares Temporal Difference Learning for the Linear Quadratic Regulatorcited by 27
  8. 2016On the low-rank approach for semidefinite programs arising in synchronization and community detectioncited by 17
  9. 2013Optimal computational and statistical rates of convergence for sparse nonconvex learning problemscited by 14
  10. 2017Boosting the Actor with Dual Criticcited by 14
  11. 2018Solving Non-Convex Non-Concave Min-Max Games Under Polyak-Lojasiewicz Conditioncited by 7
  12. 2018Online ICA: Understanding Global Dynamics of Nonconvex Optimization via Diffusion Processescited by 6
  13. 2016NESTT: A Nonconvex Primal-Dual Splitting Method for Distributed and Stochastic Optimizationcited by 5
  14. 2016Symmetry, Saddle Points, and Global Optimization Landscape of Nonconvex Matrix Factorizationcited by 3
  15. 2014Nonconvex Statistical Optimization: Minimax-Optimal Sparse PCA in Polynomial Timecited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 3; 2021: 4; 2022: 7; 2023: 1; 2024: 2; 2025: 2; 2026: 1.

20 papers.

Sort:

By area:

  1. 2021Faster Single-loop Algorithms for Minimax Optimization without Strong Concavitycited by 47
  2. 2020On Computation and Generalization of Generative Adversarial Imitation Learningcited by 27
  3. 2022Proximal Point Imitation Learningcited by 25
  4. 2022Understanding Adversarial Imitation Learning in Small Sample Regime: A Stage-coupled Analysiscited by 13
  5. 2020When Will Generative Adversarial Imitation Learning Algorithms Attain Global Convergencecited by 12
  6. 2021Provably Efficient Generative Adversarial Imitation Learning for Online and Offline Setting with Linear Function Approximationcited by 11
  7. 2022Sampling without Replacement Leads to Faster Rates in Finite-Sum Minimax Optimizationcited by 10
  8. 2022Zeroth-Order Alternating Gradient Descent Ascent Algorithms for a Class of Nonconvex-Nonconcave Minimax Problemscited by 10
  9. 2021Convex-Concave Min-Max Stackelberg Gamescited by 9
  10. 2021Efficient Performance Bounds for Primal-Dual Reinforcement Learning from Demonstrationscited by 9
  11. 2022SGDA with shuffling: faster convergence for nonconvex-PL minimax optimizationcited by 8
  12. 2020Generative Adversarial Imitation Learning with Neural Networks: Global Optimality and Convergence Ratecited by 7
8 more
  1. 2022Robust No-Regret Learning in Min-Max Stackelberg Gamescited by 5
  2. 2023Faster Stochastic Algorithms for Minimax Optimization under Polyak--Lojasiewicz Conditionscited by 4
  3. 2024C-GAIL: Stabilizing Generative Adversarial Imitation Learning with Control Theorycited by 3
  4. 2024Provably Efficient Off-Policy Adversarial Imitation Learning with Convergence Guaranteescited by 1
  5. 2025Beyond-Expert Performance with Limited Demonstrations: Efficient Imitation Learning with Double Explorationcited by 1
  6. 2025Distributionally Robust Imitation Learning: Layered Control Architecture for Certifiable Autonomycited by 1
  7. 2022Iterative Minimax Games with Coupled Linear Constraintscited by 0
  8. 2026Online Min-Max Optimization: From Individual Regrets to Cumulative Saddle Pointscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Adversarial Imitation Learning with General Function Approximation: Theoretical Analysis and Practical Algorithms
  2. 2025Convergence of Flow-Policy Gradient Learning for Linear Quadratic Regulator Problems
  3. 2020Lipschitzness Is All You Need To Tame Off-policy Generative Adversarial Imitation Learning
  4. 2026Provable Benefits of Regularization: Fast Rates for Adversarial Imitation Learning
  5. 2023Exploring Gradient Explosion in Generative Adversarial Imitation Learning: A Probabilistic Perspective
  6. 2022Robust Imitation via Mirror Descent Inverse Reinforcement Learning
  7. 2024Provably and Practically Efficient Adversarial Imitation Learning with General Function Approximation
  8. 2018Adversarial Imitation via Variational Inverse Reinforcement Learning