mlsanityData to 14 Oct 2026

RAIL: A modular framework for Reinforcement-learning-based Adversarial Imitation Learning

arXiv 2105.03756 ↗First posted 8 May 2021cs.LGcited by 2

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

While Adversarial Imitation Learning (AIL) algorithms have recently led to state-of-the-art results on various imitation learning benchmarks, it is unclear as to what impact various design decisions have on performance. To this end, we present here an organizing, modular framework called Reinforcement-learning-based Adversarial Imitation Learning (RAIL) that encompasses and generalizes a popular subclass of existing AIL approaches. Using the view espoused by RAIL, we create two new IfO (Imitation from Observation) algorithms, which we term SAIfO: SAC-based Adversarial Imitation from Observation and SILEM (Skeletal Feature Compensation for Imitation Learning with Embodiment Mismatch). We go into greater depth about SILEM in a separate technical report. In this paper, we focus on SAIfO, evaluating it on a suite of locomotion tasks from OpenAI Gym, and showing that it outperforms contemporaneous RAIL algorithms that perform IfO.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 18 of its 23 references that are in the map. First posted on arXiv 8 May 2021.

Builds on

18 of 23 references.

Closest ancestors · 5

Same direction, newest first

  1. 2021Off-Policy Imitation Learning from Observations25 Feb
  2. 2020Adversarial Soft Advantage Fitting: Imitation Learning without Policy Optimization23 Jun
  3. 2019Imitation Learning via Off-Policy Distribution Matching10 Dec
  4. 2018Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learning9 Sep
  5. 2018Generative Adversarial Imitation from Observation17 Jul

Other references · 13

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  4. 2017Proximal Policy Optimization Algorithmscited by 15,215
  5. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  6. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  7. 2014Sequence to Sequence Learning with Neural Networkscited by 2,986
  8. 2015Trust Region Policy Optimizationcited by 2,580
  9. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  10. 2016OpenAI Gymcited by 1,736
  11. 2016Generative Adversarial Imitation Learningcited by 1,597
  12. 2019AlgaeDICE: Policy Gradient from Arbitrary Experiencecited by 188
1 more
  1. 2021Skeletal Feature Compensation for Imitation Learning with Embodiment Mismatchcited by 5

Built on by

Cited by year

Citing papers per year: 2022: 2.

2 papers.

  1. 2022Adversarial Imitation Learning from Video using a State Observercited by 10
  2. 2022ABC: Adversarial Behavioral Cloning for Offline Mode-Seeking Imitation Learningcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2020Support-weighted Adversarial Imitation Learning
  2. 2026Provably Efficient Policy-Reward Co-Pretraining for Adversarial Imitation Learning
  3. 2021A Pragmatic Look at Deep Imitation Learning
  4. 2019Provably Efficient Imitation Learning from Observation Alone
  5. 2019Sample-efficient Adversarial Imitation Learning from Observation
  6. 2025SD2AIL: Adversarial Imitation Learning from Synthetic Demonstrations via Diffusion Models
  7. 2026Adversarial Imitation Learning with General Function Approximation: Theoretical Analysis and Practical Algorithms
  8. 2019Task-Relevant Adversarial Imitation Learning