C-GAIL: Stabilizing Generative Adversarial Imitation Learning with Control Theory
On the map: Offline RL and imitation learning from data → Imitation learning methods
Abstract (arXiv)
Generative Adversarial Imitation Learning (GAIL) trains a generative policy to mimic a demonstrator. It uses on-policy Reinforcement Learning (RL) to optimize a reward signal derived from a GAN-like discriminator. A major drawback of GAIL is its training instability - it inherits the complex training dynamics of GANs, and the distribution shift introduced by RL. This can cause oscillations during training, harming its sample efficiency and final policy performance. Recent work has shown that control theory can help with the convergence of a GAN's training. This paper extends this line of work, conducting a control-theoretic analysis of GAIL and deriving a novel controller that not only pushes GAIL to the desired equilibrium but also achieves asymptotic stability in a 'one-step' setting. Based on this, we propose a practical algorithm 'Controlled-GAIL' (C-GAIL). On MuJoCo tasks, our controlled variant is able to speed up the rate of convergence, reduce the range of oscillation and match the expert's distribution more closely both for vanilla GAIL and GAIL-DAC.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 15 of its 33 references that are in the map. First posted on arXiv 26 Feb 2024.
Builds on
15 of 33 references.
Closest ancestors · 8
Same direction, newest first
- 2022imitation: Clean Imitation Learning Implementations22 Nov
- 2022Minimax Optimal Online Imitation Learning via Replay Estimation30 May
- 2020Toward the Fundamental Limits of Imitation Learning13 Sep
- 2020When Will Generative Adversarial Imitation Learning Algorithms Attain Global Convergence24 Jun
- 2020On Computation and Generalization of Generative Adversarial Imitation Learning9 Jan
- 2019Wasserstein Adversarial Imitation Learning19 Jun
- 2019On the Global Convergence of Imitation Learning: A Case for Linear Quadratic Regulator11 Jan
- 2018Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learning9 Sep
Other references · 7
- 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
- 2016Generative Adversarial Imitation Learningcited by 1,576
- 2018Which Training Methods for GANs do actually Converge?cited by 672
- 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 485
- 2019Exploring the Limitations of Behavior Cloning for Autonomous Drivingcited by 343
- 2023Imitating Human Behaviour with Diffusion Modelscited by 260
- 2023Stabilizing GANs' Training with Brownian Motion Controllercited by 3
Built on by
Citing papers per year: 2024: 1; 2025: 2.
3 papers.
- 2025On Discovering Algorithms for Adversarial Imitation Learning1 Oct 2025
- 2025Model Predictive Adversarial Imitation Learning for Planning from Observation29 Jul 2025
- 2024Imitation from Diverse Behaviors: Wasserstein Quality Diversity Imitation Learning with Single-Step Archive Exploration11 Nov 2024
Similar, not cited
Close in wording, no citation either way
- 2020Generative Adversarial Imitation Learning with Neural Networks: Global Optimality and Convergence Rate
- 2018Sample-Efficient Imitation Learning via Generative Adversarial Nets
- 2023Exploring Gradient Explosion in Generative Adversarial Imitation Learning: A Probabilistic Perspective
- 2021Provably Efficient Generative Adversarial Imitation Learning for Online and Offline Setting with Linear Function Approximation
- 2020f-GAIL: Learning f-Divergence for Generative Adversarial Imitation Learning
- 2021A Pragmatic Look at Deep Imitation Learning
- 2018Risk-Sensitive Generative Adversarial Imitation Learning
- 2021GAN-Based Interactive Reinforcement Learning from Demonstration and Human Evaluative Feedback