mlsanityData to 7 Oct 2026

Adversarial Imitation Learning via Boosting

arXiv 2404.08513 ↗First posted 12 Apr 2024cs.LG cs.AIcited by 5

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Adversarial imitation learning (AIL) has stood out as a dominant framework across various imitation learning (IL) applications, with Discriminator Actor Critic (DAC) (Kostrikov et al.,, 2019) demonstrating the effectiveness of off-policy learning algorithms in improving sample efficiency and scalability to higher-dimensional observations. Despite DAC's empirical success, the original AIL objective is on-policy and DAC's ad-hoc application of off-policy training does not guarantee successful imitation (Kostrikov et al., 2019; 2020). Follow-up work such as ValueDICE (Kostrikov et al., 2020) tackles this issue by deriving a fully off-policy AIL objective. Instead in this work, we develop a novel and principled AIL algorithm via the framework of boosting. Like boosting, our new algorithm, AILBoost, maintains an ensemble of properly weighted weak learners (i.e., policies) and trains a discriminator that witnesses the maximum discrepancy between the distributions of the ensemble and the expert policy. We maintain a weighted replay buffer to represent the state-action distribution induced by the ensemble, allowing us to train discriminators using the entire data collected so far. In the weighted replay buffer, the contribution of the data from older policies are properly discounted with the weight computed based on the boosting framework. Empirically, we evaluate our algorithm on both controller state-based and pixel-based environments from the DeepMind Control Suite. AILBoost outperforms DAC on both types of environments, demonstrating the benefit of properly weighting replay buffer data for off-policy training. On state-based environments, DAC outperforms ValueDICE and IQ-Learn (Gary et al., 2021), achieving competitive performance with as little as one expert trajectory.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 27 of its 44 references that are in the map. First posted on arXiv 12 Apr 2024.

Builds on

27 of 44 references.

Closest ancestors · 10

Same direction, newest first

  1. 2023Offline Imitation Learning with Suboptimal Demonstrations via Relaxed Distribution Matching5 Mar
  2. 2021Visual Adversarial Imitation Learning using Variational Models16 Jul
  3. 2021IQ-Learn: Inverse soft-Q Learning for Imitation23 Jun
  4. 2021SoftDICE for Imitation Learning: Rethinking Off-policy Distribution Matching6 Jun
  5. 2021What Matters for Adversarial Imitation Learning?1 Jun
  6. 2021Of Moments and Matching: A Game-Theoretic Framework for Closing the Imitation Gap4 Mar
  7. 2019Imitation Learning via Off-Policy Distribution Matching10 Dec
  8. 2019Imitation Learning as f-Divergence Minimization30 May
  9. 2019Provably Efficient Imitation Learning from Observation Alone27 May
  10. 2018Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learning9 Sep

Other references · 17

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2022Generative Adversarial Networkscited by 5,603
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  4. 2015Continuous control with deep reinforcement learningcited by 3,279
  5. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  6. 2016Generative Adversarial Imitation Learningcited by 1,576
  7. 2018DeepMind Control Suitecited by 935
  8. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 781
  9. 2016f-GAN: Training Generative Neural Samplers using Variational Divergence Minimizationcited by 504
  10. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 485
  11. 2021Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learningcited by 359
  12. 2019DualDICE: Behavior-Agnostic Estimation of Discounted Stationary Distribution Correctionscited by 254
5 more
  1. 2018Provably Efficient Maximum Entropy Explorationcited by 240
  2. 2016A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Modelscited by 167
  3. 2022Hierarchical Model-Based Imitation Learning for Planning in Autonomous Drivingcited by 43
  4. 2017AdaGAN: Boosting Generative Modelscited by 34
  5. 2017Boosted Generative Modelscited by 18

Built on by

Cited by year

Citing papers per year: 2024: 2; 2025: 3.

5 papers.

  1. 2024Diffusing States and Matching Scores: A New Framework for Imitation Learningcited by 16
  2. 2025IL-SOAR : Imitation Learning with Soft Optimistic Actor cRiticcited by 5
  3. 2024Robust Deep Reinforcement Learning against Adversarial Behavior Manipulationcited by 2
  4. 2025RIZE: Adaptive Regularization for Imitation Learningcited by 0
  5. 2025Noise-Guided Transport for Imitation Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2020Off-Policy Adversarial Inverse Reinforcement Learning
  2. 2024Provably Efficient Off-Policy Adversarial Imitation Learning with Convergence Guarantees
  3. 2026Adversarial Imitation Learning with General Function Approximation: Theoretical Analysis and Practical Algorithms
  4. 2025Enabling Off-Policy Imitation Learning with Deep Actor Critic Stabilization
  5. 2023Policy Contrastive Imitation Learning
  6. 2020Support-weighted Adversarial Imitation Learning
  7. 2021A Pragmatic Look at Deep Imitation Learning
  8. 2020Non-Adversarial Imitation Learning and its Connections to Adversarial Methods