mlsanityData to 14 Oct 2026

Inverse Delayed Reinforcement Learning

arXiv 2412.02931 ↗First posted 4 Dec 2024cs.LG cs.AI cs.SYcited by 0

On the map: Offline RL and imitation learning from data → Inverse reinforcement learning

also in 1 more
Abstract (arXiv)

Inverse Reinforcement Learning (IRL) has demonstrated effectiveness in a variety of imitation tasks. In this paper, we introduce an IRL framework designed to extract rewarding features from expert trajectories affected by delayed disturbances. Instead of relying on direct observations, our approach employs an efficient off-policy adversarial training framework to derive expert features and recover optimal policies from augmented delayed observations. Empirical evaluations in the MuJoCo environment under diverse delay settings validate the effectiveness of our method. Furthermore, we provide a theoretical analysis showing that recovering expert policies from augmented delayed observations outperforms using direct delayed observations.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 36 of its 71 references that are in the map. First posted on arXiv 4 Dec 2024.

Builds on

36 of 71 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024Enhancing Inverse Reinforcement Learning through Encoding Dynamic Information in Reward Shaping4 Oct
  2. 2021Scalable Bayesian Inverse Reinforcement Learning12 Feb
  3. 2019Extrapolating Beyond Suboptimal Demonstrations via Inverse Reinforcement Learning from Observations12 Apr
  4. 2018A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progress18 Jun

Other references · 32

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2022Generative Adversarial Networkscited by 5,627
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  4. 2016Generative Adversarial Imitation Learningcited by 1,597
  5. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,103
  6. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 490
  7. 2017End-to-end Driving via Conditional Imitation Learningcited by 485
  8. 2018Lipschitz regularity of deep neural networks: analysis and efficient estimationcited by 415
  9. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 406
  10. 2020Learning Agile Robotic Locomotion Skills by Imitating Animalscited by 251
  11. 2018Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learningcited by 204
  12. 2018Generative Adversarial Imitation from Observationcited by 184
20 more
  1. 2015Maximum Entropy Deep Inverse Reinforcement Learningcited by 176
  2. 2018RUDDER: Return Decomposition for Delayed Rewardscited by 173
  3. 2017Control of a Quadrotor with Reinforcement Learningcited by 92
  4. 2017Gradient descent GAN optimization is locally stablecited by 67
  5. 2022Enforcing Hard Constraints with Soft Barriers: Safe Reinforcement Learning in Unknown Stochastic Environmentscited by 60
  6. 2018Sample-Efficient Imitation Learning via Generative Adversarial Netscited by 36
  7. 2020Delay-Aware Model-Based Reinforcement Learning for Continuous Controlcited by 34
  8. 2018Setting up a Reinforcement Learning Task with a Real-World Robotcited by 33
  9. 2020Control Frequency Adaptation via Action Persistence in Batch Reinforcement Learningcited by 28
  10. 2021Off-Policy Reinforcement Learning with Delayed Rewardscited by 25
  11. 2022Delayed Reinforcement Learning by Imitationcited by 16
  12. 2016HIRL: Hierarchical Inverse Reinforcement Learning for Long-Horizon Tasks with Delayed Rewardscited by 15
  13. 2024Reinforcement Learning from Delayed Observations via World Modelscited by 13
  14. 2022Joint Differentiable Optimization and Verification for Certified Reinforcement Learningcited by 11
  15. 2022Multi-Agent Reinforcement Learning with Reward Delayscited by 9
  16. 2023State-Wise Safe Reinforcement Learning With Pixel Observationscited by 9
  17. 2024Variational Delayed Policy Optimizationcited by 9
  18. 2018Over- and Under-Approximating Reachable Sets for Perturbed Delay Differential Equationscited by 4
  19. 2024Tree Search-Based Policy Optimization under Stochastic Execution Delaycited by 2
  20. 2024Switching Controller Synthesis for Hybrid Systems Against STL Formulascited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2019Random Expert Distillation: Imitation Learning via Expert Policy Support Estimation
  2. 2020Off-Policy Adversarial Inverse Reinforcement Learning
  3. 2022IL-flOw: Imitation Learning from Observation using Normalizing Flows
  4. 2020Inverse Reinforcement Learning from a Gradient-based Learner
  5. 2023Is Inverse Reinforcement Learning Harder than Standard Reinforcement Learning? A Theoretical Perspective
  6. 2024Non-Adversarial Inverse Reinforcement Learning via Successor Feature Matching
  7. 2023Inverse Reinforcement Learning without Reinforcement Learning
  8. 2023Adversarial Imitation Learning On Aggregated Data