# f-GAIL: Learning f-Divergence for Generative Adversarial Imitation Learning

arXiv: https://arxiv.org/abs/2010.01207 · HTML: https://mlsanity.com/paper/2010.01207/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 15 of its 32 references that are in the map.
- First posted on arXiv 2 Oct 2020.

## Placed in

- Offline RL and imitation learning from data / [Imitation learning methods](https://mlsanity.com/offlinerl/imitation-learning-methods/)

## Builds on (most cited first, 15 of 15)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [High-Dimensional Continuous Control Using Generalized Advantage Estimation](https://mlsanity.com/paper/1506.02438/) (2015)
- [Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1606.03476/) (2016)
- [Wasserstein GAN](https://mlsanity.com/paper/1701.07875/) (2017)
- [f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization](https://mlsanity.com/paper/1606.00709/) (2016)
- [Learning Robust Rewards with Adversarial Inverse Reinforcement Learning](https://mlsanity.com/paper/1710.11248/) (2017)
- [Input Convex Neural Networks](https://mlsanity.com/paper/1609.07152/) (2016)
- [A Divergence Minimization Perspective on Imitation Learning Methods](https://mlsanity.com/paper/1911.02256/) (2019)
- [A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Models](https://mlsanity.com/paper/1611.03852/) (2016)
- [Imitation Learning as f-Divergence Minimization](https://mlsanity.com/paper/1905.12888/) (2019)
- [Training Deep Energy-Based Models with f-Divergence Minimization](https://mlsanity.com/paper/2003.03463/) (2020)
- [Wasserstein Variational Inference](https://mlsanity.com/paper/1805.11284/) (2018)
- [Augmenting GAIL with BC for sample efficient imitation learning](https://mlsanity.com/paper/2001.07798/) (2020)
- [Reparameterized Variational Divergence Minimization for Stable Imitation](https://mlsanity.com/paper/2006.10810/) (2020)

## Built on by (most cited first, 20 of 20)

- [Planning for Sample Efficient Imitation Learning](https://mlsanity.com/paper/2210.09598/) (2022)
- [DiffAIL: Diffusion Adversarial Imitation Learning](https://mlsanity.com/paper/2312.06348/) (2023)
- [Diffusing States and Matching Scores: A New Framework for Imitation Learning](https://mlsanity.com/paper/2410.13855/) (2024)
- [Internally Rewarded Reinforcement Learning](https://mlsanity.com/paper/2302.00270/) (2023)
- [Inverse Contextual Bandits: Learning How Behavior Evolves over Time](https://mlsanity.com/paper/2107.06317/) (2021)
- [Inverse Decision Modeling: Learning Interpretable Representations of Behavior](https://mlsanity.com/paper/2310.18591/) (2023)
- [Time-series Generation by Contrastive Imitation](https://mlsanity.com/paper/2311.01388/) (2023)
- [Inferring Versatile Behavior from Demonstrations by Matching Geometric Descriptors](https://mlsanity.com/paper/2210.08121/) (2022)
- [Faster Algorithms for Learning Convex Functions](https://mlsanity.com/paper/2111.01348/) (2021)
- [ARC - Actor Residual Critic for Adversarial Imitation Learning](https://mlsanity.com/paper/2206.02095/) (2022)
- [R2-Diff: Denoising by diffusion as a refinement of retrieved motion for image-based motion prediction](https://mlsanity.com/paper/2306.09483/) (2023)
- [Off-policy Imitation Learning from Visual Inputs](https://mlsanity.com/paper/2111.04345/) (2021)
- [A Unified Framework for Diffusion Model Unlearning with f-Divergence](https://mlsanity.com/paper/2509.21167/) (2025)
- [Auto-Encoding Adversarial Imitation Learning](https://mlsanity.com/paper/2206.11004/) (2022)
- [Quantum Imitation Learning](https://mlsanity.com/paper/2304.02480/) (2023)
- [PAIL: Performance based Adversarial Imitation Learning Engine for Carbon Neutral Optimization](https://mlsanity.com/paper/2407.08910/) (2024)
- [On Discovering Algorithms for Adversarial Imitation Learning](https://mlsanity.com/paper/2510.00922/) (2025)
- [Understanding Hindsight Goal Relabeling from a Divergence Minimization Perspective](https://mlsanity.com/paper/2209.13046/) (2022)
- [Sample-efficient Adversarial Imitation Learning](https://mlsanity.com/paper/2303.07846/) (2023)
- [Long-Horizon Language Model Reinforcement Learning via Progressive Point Matching](https://mlsanity.com/paper/2609.07303/) (2026)

## Abstract (arXiv)

Imitation learning (IL) aims to learn a policy from expert demonstrations that minimizes the discrepancy between the learner and expert behaviors. Various imitation learning algorithms have been proposed with different pre-determined divergences to quantify the discrepancy. This naturally gives rise to the following question: Given a set of expert demonstrations, which divergence can recover the expert policy more accurately with higher data efficiency? In this work, we propose f-GAIL, a new generative adversarial imitation learning (GAIL) model, that automatically learns a discrepancy measure from the f-divergence family as well as a policy capable of producing expert-like behaviors. Compared with IL baselines with various predefined divergence measures, f-GAIL learns better policies with higher data efficiency in six physics-based control tasks.