mlsanityData to 7 Oct 2026

Understanding Adversarial Imitation Learning in Small Sample Regime: A Stage-coupled Analysis

arXiv 2208.01899 ↗First posted 3 Aug 2022cs.LGcited by 13

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Imitation learning learns a policy from expert trajectories. While the expert data is believed to be crucial for imitation quality, it was found that a kind of imitation learning approach, adversarial imitation learning (AIL), can have exceptional performance. With as little as only one expert trajectory, AIL can match the expert performance even in a long horizon, on tasks such as locomotion control. There are two mysterious points in this phenomenon. First, why can AIL perform well with only a few expert trajectories? Second, why does AIL maintain good performance despite the length of the planning horizon? In this paper, we theoretically explore these two questions. For a total-variation-distance-based AIL (called TV-AIL), our analysis shows a horizon-free imitation gap mathcal O(min1, √|mathcal S|/N ) on a class of instances abstracted from locomotion control tasks. Here |mathcal S| is the state space size for a tabular Markov decision process, and N is the number of expert trajectories. We emphasize two important features of our bound. First, this bound is meaningful in both small and large sample regimes. Second, this bound suggests that the imitation gap of TV-AIL is at most 1 regardless of the planning horizon. Therefore, this bound can explain the empirical observation. Technically, we leverage the structure of multi-stage policy optimization in TV-AIL and present a new stage-coupled analysis via dynamic programming

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 28 of its 63 references that are in the map. First posted on arXiv 3 Aug 2022.

Builds on

28 of 63 references.

Closest ancestors · 19

Same direction, newest first

  1. 2022Rethinking ValueDice: Does It Really Improve Performance?5 Feb
  2. 2021Provably Efficient Generative Adversarial Imitation Learning for Online and Offline Setting with Linear Function Approximation19 Aug
  3. 2021What Matters for Adversarial Imitation Learning?1 Jun
  4. 2021Of Moments and Matching: A Game-Theoretic Framework for Closing the Imitation Gap4 Mar
  5. 2021Provably Breaking the Quadratic Error Compounding Barrier in Imitation Learning, Optimally25 Feb
  6. 2020Error Bounds of Imitating Policies and Environments22 Oct
  7. 2020Toward the Fundamental Limits of Imitation Learning13 Sep
  8. 2020Intrinsic Reward Driven Imitation Learning via Generative Model26 Jun
  9. 2020Primal Wasserstein Imitation Learning8 Jun
  10. 2020Energy-Based Imitation Learning20 Apr
  11. 2020On Computation and Generalization of Generative Adversarial Imitation Learning9 Jan
  12. 2019Imitation Learning via Off-Policy Distribution Matching10 Dec
7 more
  1. 2019A Divergence Minimization Perspective on Imitation Learning Methods6 Nov
  2. 2019Imitation Learning from Pixel-Level Demonstrations by HashReward9 Sep
  3. 2019Imitation Learning as f-Divergence Minimization30 May
  4. 2019Provably Efficient Imitation Learning from Observation Alone27 May
  5. 2019On the Global Convergence of Imitation Learning: A Case for Linear Quadratic Regulator11 Jan
  6. 2018An Algorithmic Perspective on Imitation Learning16 Nov
  7. 2018Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learning9 Sep

Other references · 9

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  2. 2016Generative Adversarial Imitation Learningcited by 1,576
  3. 2015End-to-End Training of Deep Visuomotor Policiescited by 847
  4. 2019Provably Efficient Reinforcement Learning with Linear Function Approximationcited by 514
  5. 2019Online Learning: A Modern Introduction Using Convex Optimizationcited by 407
  6. 2019Information-Theoretic Considerations in Batch Reinforcement Learningcited by 285
  7. 2020Reward-Free Exploration for Reinforcement Learningcited by 177
  8. 2019On the Theory of Policy Gradient Methods: Optimality, Approximation, and Distribution Shiftcited by 174
  9. 2020Fast active learning for pure exploration in reinforcement learningcited by 92

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 2; 2025: 3; 2026: 4.

13 papers.

Sort:

By area:

  1. 2023Policy Optimization in RLHF: The Impact of Out-of-preference Datacited by 35
  2. 2025Generalist Reward Models: Found Inside Large Language Modelscited by 34
  3. 2023Provably Efficient Adversarial Imitation Learning with Unknown Transitionscited by 16
  4. 2023Reward-Consistent Dynamics Models are Strongly Generalizable for Offline Reinforcement Learningcited by 12
  5. 2025Optimistically Optimistic Exploration for Provably Efficient Infinite-Horizon Reinforcement and Imitation Learningcited by 12
  6. 2024OLLIE: Imitation Learning from Offline Pretraining to Online Finetuningcited by 6
  7. 2024Provably and Practically Efficient Adversarial Imitation Learning with General Function Approximationcited by 6
  8. 2023Theoretical Analysis of Offline Imitation With Supplementary Datasetcited by 2
  9. 2025Near-Optimal Second-Order Guarantees for Model-Based Adversarial Imitation Learningcited by 2
  10. 2026Adversarial Imitation Learning with General Function Approximation: Theoretical Analysis and Practical Algorithmscited by 1
  11. 2026Non-Adversarial Imitation Learning Provably Free of Compounding Errors: The Value Flow Mechanismcited by 0
  12. 2026Provably Efficient Policy-Reward Co-Pretraining for Adversarial Imitation Learningcited by 0
1 more
  1. 2026Improving Online Reinforcement Learning via Bidirectional Behavior Prior Distillationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2021On Generalization of Adversarial Imitation Learning and Beyond
  2. 2021A Pragmatic Look at Deep Imitation Learning
  3. 2024Provably Efficient Off-Policy Adversarial Imitation Learning with Convergence Guarantees
  4. 2024Offline Imitation Learning by Controlling the Effective Planning Horizon
  5. 2020State-only Imitation with Transition Dynamics Mismatch
  6. 2020Support-weighted Adversarial Imitation Learning
  7. 2026Provable Benefits of Regularization: Fast Rates for Adversarial Imitation Learning
  8. 2022Imitation Learning by State-Only Distribution Matching