mlsanityData to 7 Oct 2026

Provably and Practically Efficient Adversarial Imitation Learning with General Function Approximation

arXiv 2411.00610 ↗First posted 1 Nov 2024cs.LGcited by 6

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

As a prominent category of imitation learning methods, adversarial imitation learning (AIL) has garnered significant practical success powered by neural network approximation. However, existing theoretical studies on AIL are primarily limited to simplified scenarios such as tabular and linear function approximation and involve complex algorithmic designs that hinder practical implementation, highlighting a gap between theory and practice. In this paper, we explore the theoretical underpinnings of online AIL with general function approximation. We introduce a new method called optimization-based AIL (OPT-AIL), which centers on performing online optimization for reward functions and optimism-regularized Bellman error minimization for Q-value functions. Theoretically, we prove that OPT-AIL achieves polynomial expert sample complexity and interaction complexity for learning near-expert policies. To our best knowledge, OPT-AIL is the first provably efficient AIL method with general function approximation. Practically, OPT-AIL only requires the approximate optimization of two objectives, thereby facilitating practical implementation. Empirical studies demonstrate that OPT-AIL outperforms previous state-of-the-art deep AIL methods in several challenging tasks.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 47 of its 66 references that are in the map. First posted on arXiv 1 Nov 2024.

Builds on

47 of 66 references.

Closest ancestors · 16

Same direction, newest first

  1. 2024Is Behavior Cloning All You Need? Understanding Horizon in Imitation Learning20 Jul
  2. 2023Provably Efficient Adversarial Imitation Learning with Unknown Transitions11 Jun
  3. 2022Proximal Point Imitation Learning22 Sep
  4. 2022Understanding Adversarial Imitation Learning in Small Sample Regime: A Stage-coupled Analysis3 Aug
  5. 2022Minimax Optimal Online Imitation Learning via Replay Estimation30 May
  6. 2021Provably Efficient Generative Adversarial Imitation Learning for Online and Offline Setting with Linear Function Approximation19 Aug
  7. 2021IQ-Learn: Inverse soft-Q Learning for Imitation23 Jun
  8. 2020Error Bounds of Imitating Policies and Environments22 Oct
  9. 2020Toward the Fundamental Limits of Imitation Learning13 Sep
  10. 2020On Computation and Generalization of Generative Adversarial Imitation Learning9 Jan
  11. 2019Imitation Learning via Off-Policy Distribution Matching10 Dec
  12. 2019A Divergence Minimization Perspective on Imitation Learning Methods6 Nov
4 more
  1. 2019Imitation Learning as f-Divergence Minimization30 May
  2. 2019Provably Efficient Imitation Learning from Observation Alone27 May
  3. 2018Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learning9 Sep
  4. 2018Behavioral Cloning from Observation4 May

Other references · 31

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  2. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  3. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  4. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  5. 2016Generative Adversarial Imitation Learningcited by 1,576
  6. 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
  7. 2018DeepMind Control Suitecited by 935
  8. 2019Introduction to Online Convex Optimizationcited by 793
  9. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 753
  10. 2017Spectrally-normalized margin bounds for neural networkscited by 681
  11. 2019Provably Efficient Reinforcement Learning with Linear Function Approximationcited by 514
  12. 2019Online Learning: A Modern Introduction Using Convex Optimizationcited by 407
19 more
  1. 2021Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learningcited by 359
  2. 2016Contextual Decision Processes with Low Bellman Rank are PAC-Learnablecited by 290
  3. 2021Bellman Eluder Dimension: New Rich Classes of RL Problems, and Sample-Efficient Algorithmscited by 222
  4. 2014Taming the Monster: A Fast and Simple Algorithm for Contextual Banditscited by 200
  5. 2014Model-based Reinforcement Learning and the Eluder Dimensioncited by 116
  6. 2022Adversarially Trained Actor Critic for Offline Reinforcement Learningcited by 111
  7. 2018Virtual-Taobao: Virtualizing Real-world Online Retail Environment for Reinforcement Learningcited by 80
  8. 2018Generative Adversarial User Model for Reinforcement Learning Based Recommendation Systemcited by 64
  9. 2023Inverse Reinforcement Learning without Reinforcement Learningcited by 35
  10. 2019Online Non-Convex Learning: Following the Perturbed Leader is Optimalcited by 31
  11. 2023Adversarial Model for Offline Reinforcement Learningcited by 31
  12. 2024Hybrid Inverse Reinforcement Learningcited by 31
  13. 2021Online Apprenticeship Learningcited by 28
  14. 2022From Dirichlet to Rubin: Optimistic Exploration in RL without Bonusescited by 22
  15. 2023Maximize to Explore: One Objective Function Fusing Estimation, Planning, and Explorationcited by 20
  16. 2023Robust Lipschitz Bandits to Adversarial Corruptionscited by 15
  17. 2024Harnessing Density Ratios for Online Reinforcement Learningcited by 15
  18. 2022Transferable Reward Learning by Dynamics-Agnostic Discriminator Ensemblecited by 5
  19. 2022A Note on Target Q-learning For Solving Finite MDPs with A Generative Oraclecited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 4.

6 papers.

  1. 2025IL-SOAR : Imitation Learning with Soft Optimistic Actor cRiticcited by 5
  2. 2025Near-Optimal Second-Order Guarantees for Model-Based Adversarial Imitation Learningcited by 2
  3. 2026Your Self-Play Algorithm is Secretly an Adversarial Imitator: Understanding LLM Self-Play through the Lens of Imitation Learningcited by 1
  4. 2026Adversarial Imitation Learning with General Function Approximation: Theoretical Analysis and Practical Algorithmscited by 1
  5. 2026Non-Adversarial Imitation Learning Provably Free of Compounding Errors: The Value Flow Mechanismcited by 0
  6. 2026Provable Benefits of Regularization: Fast Rates for Adversarial Imitation Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2021On Generalization of Adversarial Imitation Learning and Beyond
  2. 2024Provably Efficient Off-Policy Adversarial Imitation Learning with Convergence Guarantees
  3. 2022Planning for Sample Efficient Imitation Learning
  4. 2026Provably Efficient Policy-Reward Co-Pretraining for Adversarial Imitation Learning
  5. 2020Generative Adversarial Imitation Learning with Neural Networks: Global Optimality and Convergence Rate
  6. 2020Off-Policy Adversarial Inverse Reinforcement Learning
  7. 2018Sample-Efficient Imitation Learning via Generative Adversarial Nets
  8. 2020Wasserstein Distance guided Adversarial Imitation Learning with Reward Shape Exploration