mlsanityData to 14 Oct 2026

Recursive Deep Inverse Reinforcement Learning

arXiv 2504.13241 ↗First posted 17 Apr 2025cs.LG cs.AIcited by 0

On the map: Offline RL and imitation learning from data → Inverse reinforcement learning

also in 1 more
Abstract (arXiv)

Inferring an adversary's goals from exhibited behavior is crucial for counterplanning and non-cooperative multi-agent systems in domains like cybersecurity, military, and strategy games. Deep Inverse Reinforcement Learning (IRL) methods based on maximum entropy principles show promise in recovering adversaries goals but are typically offline, require large batch sizes with gradient descent, and rely on first-order updates, limiting their applicability in real-time scenarios. We propose an online Recursive Deep Inverse Reinforcement Learning (RDIRL) approach to recover the cost function governing the adversary actions and goals. Specifically, we minimize an upper bound on the standard Guided Cost Learning (GCL) objective using sequential second-order Newton updates, akin to the Extended Kalman Filter (EKF), leading to a fast (in terms of convergence) learning algorithm. We demonstrate that RDIRL is able to recover cost and reward functions of expert agents in standard and adversarial benchmark tasks. Experiments on benchmark tasks show that our proposed approach outperforms several leading IRL algorithms.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 30 of its 54 references that are in the map. First posted on arXiv 17 Apr 2025.

Builds on

30 of 54 references.

Closest ancestors · 8

Same direction, newest first

  1. 2022Maximum-Likelihood Inverse Reinforcement Learning with Finite-Time Guarantees4 Oct
  2. 2022Structural Estimation of Markov Decision Processes in High-Dimensional State Space with Finite-Time Guarantees4 Oct
  3. 2022Receding Horizon Inverse Reinforcement Learning9 Jun
  4. 2020Online Observer-Based Inverse Reinforcement Learning3 Nov
  5. 2020Online inverse reinforcement learning with limited data18 Aug
  6. 2020Online inverse reinforcement learning with unknown disturbances9 Mar
  7. 2019Meta-Inverse Reinforcement Learning with Probabilistic Context Variables20 Sep
  8. 2018Learning a Prior over Intent via Meta-Inverse Reinforcement Learning31 May

Other references · 22

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2016OpenAI Gymcited by 1,736
  4. 2016Generative Adversarial Imitation Learningcited by 1,597
  5. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 490
  6. 2018Shampoo: Preconditioned Stochastic Tensor Optimizationcited by 426
  7. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 406
  8. 2016Cooperative Inverse Reinforcement Learningcited by 300
  9. 2015Maximum Entropy Deep Inverse Reinforcement Learningcited by 176
  10. 2016A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Modelscited by 168
  11. 2021Of Moments and Matching: A Game-Theoretic Framework for Closing the Imitation Gapcited by 81
  12. 2019From Language to Goals: Inverse Reinforcement Learning for Vision-Based Instruction Followingcited by 67
10 more
  1. 2019Pontryagin Differentiable Programming: An End-to-End Learning and Control Frameworkcited by 52
  2. 2020Efficient Sampling-Based Maximum Entropy Inverse Reinforcement Learning with Application to Autonomous Drivingcited by 34
  3. 2017Block-diagonal Hessian-free Optimization for Training Neural Networkscited by 10
  4. 2024Learning Physics Informed Neural ODEs With Partial Measurementscited by 9
  5. 2020Derivation of a Constant Velocity Motion Model for Visual Trackingcited by 6
  6. 2020Online Inverse Optimal Control for Control-Constrained Discrete-Time Systems on Finite and Infinite Horizonscited by 5
  7. 2022Hybrid Neural Network Augmented Physics-based Models for Nonlinear Filteringcited by 5
  8. 2014Inverse optimal control with polynomial optimizationcited by 2
  9. 2024Extended Kalman Filtering for Recursive Online Discrete-Time Inverse Optimal Controlcited by 1
  10. 2024Continuously Optimizing Radar Placement with Model Predictive Path Integralscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2020Scalable Multi-Agent Inverse Reinforcement Learning via Actor-Attention-Critic
  2. 2024Non-Adversarial Inverse Reinforcement Learning via Successor Feature Matching
  3. 2018A Framework and Method for Online Inverse Reinforcement Learning
  4. 2023Is Inverse Reinforcement Learning Harder than Standard Reinforcement Learning? A Theoretical Perspective
  5. 2019Multi-Agent Adversarial Inverse Reinforcement Learning
  6. 2021A Generalised Inverse Reinforcement Learning Framework
  7. 2020oIRL: Robust Adversarial Inverse Reinforcement Learning with Temporally Extended Actions
  8. 2018A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progress