mlsanityData to 7 Oct 2026

Enabling Off-Policy Imitation Learning with Deep Actor Critic Stabilization

arXiv 2511.07288 ↗First posted 10 Nov 2025cs.LG cs.AIcited by 0

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Learning complex policies with Reinforcement Learning (RL) is often hindered by instability and slow convergence, a problem exacerbated by the difficulty of reward engineering. Imitation Learning (IL) from expert demonstrations bypasses this reliance on rewards. However, state-of-the-art IL methods, exemplified by Generative Adversarial Imitation Learning (GAIL)Ho et. al, suffer from severe sample inefficiency. This is a direct consequence of their foundational on-policy algorithms, such as TRPO Schulman et.al. In this work, we introduce an adversarial imitation learning algorithm that incorporates off-policy learning to improve sample efficiency. By combining an off-policy framework with auxiliary techniques specifically, in this case a double Q network based stabilization and value learning without reward function inference we demonstrate a reduction in the samples required to robustly match expert behavior.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 11 of its 19 references that are in the map. First posted on arXiv 10 Nov 2025.

Builds on

11 of 19 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Imitation Learning from Suboptimal Demonstrations via Meta-Learning An Action Ranker28 Dec
  2. 2018An Algorithmic Perspective on Imitation Learning16 Nov

Other references · 9

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2022Generative Adversarial Networkscited by 5,603
  3. 2015Continuous control with deep reinforcement learningcited by 3,279
  4. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  5. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  6. 2016Generative Adversarial Imitation Learningcited by 1,576
  7. 2017Hindsight Experience Replaycited by 1,096
  8. 2017Parameter Space Noise for Explorationcited by 158
  9. 2016Q-Prop: Sample-Efficient Policy Gradient with An Off-Policy Criticcited by 77

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Provably Efficient Off-Policy Adversarial Imitation Learning with Convergence Guarantees
  2. 2019Random Expert Distillation: Imitation Learning via Expert Policy Support Estimation
  3. 2020Off-Policy Adversarial Inverse Reinforcement Learning
  4. 2019A New Framework for Query Efficient Active Imitation Learning
  5. 2018Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learning
  6. 2021A Pragmatic Look at Deep Imitation Learning
  7. 2018Sample-Efficient Imitation Learning via Generative Adversarial Nets
  8. 2021Imitation Learning by Reinforcement Learning