mlsanityData to 7 Oct 2026

On Value Discrepancy of Imitation Learning

arXiv 1911.07027 ↗First posted 16 Nov 2019cs.LG cs.AI stat.MLcited by 4

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Imitation learning trains a policy from expert demonstrations. Imitation learning approaches have been designed from various principles, such as behavioral cloning via supervised learning, apprenticeship learning via inverse reinforcement learning, and GAIL via generative adversarial learning. In this paper, we propose a framework to analyze the theoretical property of imitation learning approaches based on discrepancy propagation analysis. Under the infinite-horizon setting, the framework leads to the value discrepancy of behavioral cloning in an order of O((1-γ)-2). We also show that the framework leads to the value discrepancy of GAIL in an order of O((1-γ)-1). It implies that GAIL has less compounding errors than behavioral cloning, which is also verified empirically in this paper. To the best of our knowledge, we are the first one to analyze GAIL's performance theoretically. The above results indicate that the proposed framework is a general tool to analyze imitation learning approaches. We hope our theoretical results can provide insights for future improvements in imitation learning algorithms.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 12 of its 28 references that are in the map. First posted on arXiv 16 Nov 2019.

Builds on

12 of 28 references.

Closest ancestors · 2

Same direction, newest first

  1. 2018Variational Discriminator Bottleneck: Improving Imitation Learning, Inverse RL, and GANs by Constraining Information Flow1 Oct
  2. 2018Behavioral Cloning from Observation4 May

Other references · 10

  1. 2016OpenAI Gymcited by 1,684
  2. 2016Generative Adversarial Imitation Learningcited by 1,576
  3. 2016End to End Learning for Self-Driving Carscited by 1,000
  4. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 485
  5. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 400
  6. 2017Generalization and Equilibrium in Generative Adversarial Nets (GANs)cited by 165
  7. 2015How (not) to Train your Generative Model: Scheduled Sampling, Likelihood, Adversary?cited by 76
  8. 2016Model-Free Imitation Learning with Policy Optimizationcited by 52
  9. 2017On the Discrimination-Generalization Tradeoff in GANscited by 48
  10. 2019On the Correctness and Sample Complexity of Inverse Reinforcement Learningcited by 8

Built on by

Cited by year

Citing papers per year: 2021: 2; 2022: 0; 2023: 1; 2024: 1.

4 papers.

  1. 2023Guarded Policy Optimization with Imperfect Online Demonstrationscited by 10
  2. 2024Trustworthy Human-AI Collaboration: Reinforcement Learning with Human Feedback and Physics Knowledge for Safe Autonomous Drivingcited by 8
  3. 2021Adversarially Regularized Policy Learning Guided by Trajectory Optimizationcited by 4
  4. 2021Curriculum Offline Imitation Learningcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2018Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learning
  2. 2022Causal Imitation Learning with Unobserved Confounders
  3. 2020On Computation and Generalization of Generative Adversarial Imitation Learning
  4. 2021Imitation Learning by Reinforcement Learning
  5. 2019Random Expert Distillation: Imitation Learning via Expert Policy Support Estimation
  6. 2022Rethinking ValueDice: Does It Really Improve Performance?
  7. 2021Deterministic and Discriminative Imitation (D2-Imitation): Revisiting Adversarial Imitation for Sample Efficiency
  8. 2019SQIL: Imitation Learning via Reinforcement Learning with Sparse Rewards