mlsanityData to 14 Oct 2026

Near-Optimal Second-Order Guarantees for Model-Based Adversarial Imitation Learning

arXiv 2510.09487 ↗First posted 10 Oct 2025cs.LGcited by 2

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

We study online adversarial imitation learning (AIL), where an agent learns from offline expert demonstrations and interacts with the environment online without access to rewards. Despite strong empirical results, the benefits of online interaction and the impact of stochasticity remain poorly understood. We address these gaps by introducing a model-based AIL algorithm (MB-AIL) and establish its horizon-free, second-order sample-complexity guarantees under general function approximations for both expert data and reward-free interactions. These second-order bounds provide an instance-dependent result that can scale with the variance of returns under the relevant policies and therefore tighten as the system approaches determinism. Together with second-order, information-theoretic lower bounds on a newly constructed hard-instance family, we show that MB-AIL attains minimax-optimal sample complexity for online interaction (up to logarithmic factors) with limited expert demonstrations and matches the lower bound for expert demonstrations in terms of the dependence on horizon H, precision ε and the policy variance σ2. Experiments further validate our theoretical findings and demonstrate that a practical implementation of MB-AIL matches or surpasses the sample efficiency of existing methods.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 47 of its 60 references that are in the map. First posted on arXiv 10 Oct 2025.

Builds on

47 of 60 references.

Closest ancestors · 16

Same direction, newest first

  1. 2025Inverse Q-Learning Done Right: Offline Imitation Learning in Qπ-Realizable MDPs26 May
  2. 2024Provably and Practically Efficient Adversarial Imitation Learning with General Function Approximation1 Nov
  3. 2024Reward-free World Models for Online Imitation Learning17 Oct
  4. 2024Is Behavior Cloning All You Need? Understanding Horizon in Imitation Learning20 Jul
  5. 2024Provably Efficient Off-Policy Adversarial Imitation Learning with Convergence Guarantees26 May
  6. 2024Efficient Imitation Learning with Conservative World Models21 May
  7. 2024Imitation Learning in Discounted Linear MDPs without exploration assumptions3 May
  8. 2023Provably Efficient Adversarial Imitation Learning with Unknown Transitions11 Jun
  9. 2022Planning for Sample Efficient Imitation Learning18 Oct
  10. 2022Understanding Adversarial Imitation Learning in Small Sample Regime: A Stage-coupled Analysis3 Aug
  11. 2021Implicit Behavioral Cloning1 Sep
  12. 2021Provably Efficient Generative Adversarial Imitation Learning for Online and Offline Setting with Linear Function Approximation19 Aug
4 more
  1. 2021IQ-Learn: Inverse soft-Q Learning for Imitation23 Jun
  2. 2020Toward the Fundamental Limits of Imitation Learning13 Sep
  3. 2020On Computation and Generalization of Generative Adversarial Imitation Learning9 Jan
  4. 2019Provably Efficient Imitation Learning from Observation Alone27 May

Other references · 31

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  2. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,838
  3. 2016Generative Adversarial Imitation Learningcited by 1,597
  4. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 773
  5. 2020MOPO: Model-based Offline Policy Optimizationcited by 636
  6. 2016Contextual Decision Processes with Low Bellman Rank are PAC-Learnablecited by 292
  7. 2021Bellman Eluder Dimension: New Rich Classes of RL Problems, and Sample-Efficient Algorithmscited by 223
  8. 2021The Statistical Complexity of Interactive Decision Makingcited by 218
  9. 2019Tighter Problem-Dependent Regret Bounds in Reinforcement Learning without Domain Knowledge using Value Function Boundscited by 195
  10. 2021Pessimistic Model-based Offline Reinforcement Learning under Partial Coveragecited by 164
  11. 2018Model-Based Value Estimation for Efficient Model-Free Reinforcement Learningcited by 156
  12. 2024PLUTO: Pushing the Limit of Imitation Learning-based Planning for Autonomous Drivingcited by 84
19 more
  1. 2023Waypoint-Based Imitation Learning for Robotic Manipulationcited by 61
  2. 2022GEC: A Unified Framework for Interactive Decision Making in MDP, POMDP, and Beyondcited by 46
  3. 2023Settling the Sample Complexity of Online Reinforcement Learningcited by 45
  4. 2022VOQL: Towards Optimal Regret in Model-free RL with Nonlinear Function Approximationcited by 44
  5. 2018Variance-Aware Regret Bounds for Undiscounted Reinforcement Learning in MDPscited by 43
  6. 2022PAC Reinforcement Learning for Predictive State Representationscited by 43
  7. 2023Variance-Dependent Regret Bounds for Linear Bandits and Reinforcement Learning: Adaptivity and Computational Efficiencycited by 40
  8. 2022A General Framework for Sample-Efficient Function Approximation in Reinforcement Learningcited by 31
  9. 2023Corruption-Robust Offline Reinforcement Learning with General Function Approximationcited by 31
  10. 2016Model-based Adversarial Imitation Learningcited by 29
  11. 2021Online Apprenticeship Learningcited by 29
  12. 2023The Benefits of Being Distributional: Small-Loss Bounds for Reinforcement Learningcited by 27
  13. 2023Sharp Variance-Dependent Bounds in Reinforcement Learning: Best of Both Worlds in Stochastic and Deterministic Environmentscited by 19
  14. 2021PC-MLP: Model-based Reinforcement Learning with Policy Cover Guided Explorationcited by 14
  15. 2025Optimistically Optimistic Exploration for Provably Efficient Infinite-Horizon Reinforcement and Imitation Learningcited by 12
  16. 2024Model-based RL as a Minimalist Approach to Horizon-Free and Second-Order Boundscited by 11
  17. 2024Model-based Reinforcement Learning for Parameterized Action Spacescited by 8
  18. 2024Uncertainty-Aware Reward-Free Exploration with General Function Approximationcited by 5
  19. 2021Generative Adversarial Imitation Learning for End-to-End Autonomous Driving on Urban Environmentscited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Your Self-Play Algorithm is Secretly an Adversarial Imitator: Understanding LLM Self-Play through the Lens of Imitation Learningcited by 1
  2. 2026Provable Benefits of Regularization: Fast Rates for Adversarial Imitation Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Adversarial Imitation Learning with General Function Approximation: Theoretical Analysis and Practical Algorithms
  2. 2021On Generalization of Adversarial Imitation Learning and Beyond
  3. 2020Explaining Fast Improvement in Online Imitation Learning
  4. 2026Non-Adversarial Imitation Learning Provably Free of Compounding Errors: The Value Flow Mechanism
  5. 2022Minimax Optimal Online Imitation Learning via Replay Estimation
  6. 2026Latent Wasserstein Adversarial Imitation Learning
  7. 2026Provably Efficient Policy-Reward Co-Pretraining for Adversarial Imitation Learning
  8. 2021Deterministic and Discriminative Imitation (D2-Imitation): Revisiting Adversarial Imitation for Sample Efficiency