mlsanityData to 7 Oct 2026

Receding Horizon Inverse Reinforcement Learning

arXiv 2206.04477 ↗First posted 9 Jun 2022cs.LG cs.AIcited by 10

On the map: Offline RL and imitation learning from data → Inverse reinforcement learning

Abstract (arXiv)

Inverse reinforcement learning (IRL) seeks to infer a cost function that explains the underlying goals and preferences of expert demonstrations. This paper presents receding horizon inverse reinforcement learning (RHIRL), a new IRL algorithm for high-dimensional, noisy, continuous systems with black-box dynamic models. RHIRL addresses two key challenges of IRL: scalability and robustness. To handle high-dimensional continuous systems, RHIRL matches the induced optimal trajectories with expert demonstrations locally in a receding horizon manner and 'stitches' together the local solutions to learn the cost; it thereby avoids the 'curse of dimensionality'. This contrasts sharply with earlier algorithms that match with expert demonstrations globally over the entire high-dimensional state space. To be robust against imperfect expert demonstrations and control noise, RHIRL learns a state-dependent cost function 'disentangled' from system dynamics under mild conditions. Experiments on benchmark tasks show that RHIRL outperforms several leading IRL algorithms in most instances. We also prove that the cumulative error of RHIRL grows linearly with the task duration.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 24 of its 48 references that are in the map. First posted on arXiv 9 Jun 2022.

Builds on

24 of 48 references.

Closest ancestors · 4

Same direction, newest first

  1. 2021OPIRL: Sample Efficient Off-Policy Inverse Reinforcement Learning via Distribution Matching9 Sep
  2. 2020f-IRL: Inverse Reinforcement Learning via State Marginal Matching9 Nov
  3. 2019Extrapolating Beyond Suboptimal Demonstrations via Inverse Reinforcement Learning from Observations12 Apr
  4. 2018Adversarial Imitation via Variational Inverse Reinforcement Learning17 Sep

Other references · 20

  1. 2022Generative Adversarial Networkscited by 5,603
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  4. 2016OpenAI Gymcited by 1,684
  5. 2016Generative Adversarial Imitation Learningcited by 1,576
  6. 2018World Modelscited by 1,444
  7. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 485
  8. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 400
  9. 2018Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learningcited by 203
  10. 2019A Divergence Minimization Perspective on Imitation Learning Methodscited by 188
  11. 2019Efficient Exploration via State Marginal Matchingcited by 186
  12. 2015Maximum Entropy Deep Inverse Reinforcement Learningcited by 173
8 more
  1. 2016A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Modelscited by 167
  2. 2019Imitation Learning as f-Divergence Minimizationcited by 121
  3. 2019Imitation Learning from Imperfect Demonstrationcited by 112
  4. 2021Reward (Mis)design for Autonomous Drivingcited by 102
  5. 2020Learning from Suboptimal Demonstration via Self-Supervised Reward Regressioncited by 84
  6. 2021Feedback in Imitation Learning: The Three Regimes of Covariate Shiftcited by 68
  7. 2017Learning a visuomotor controller for real world robotic grasping using simulated depth imagescited by 29
  8. 2022Spatiotemporal Costmap Inference for MPC via Deep Inverse Reinforcement Learningcited by 10

Built on by

Cited by year

Citing papers per year: 2023: 3; 2024: 6; 2025: 1.

10 papers.

  1. 2024In-Trajectory Inverse Reinforcement Learning: Learn Incrementally Before An Ongoing Trajectory Terminatescited by 7
  2. 2023Learning Reward for Physical Skills using Large Language Modelcited by 5
  3. 2024Language-Model-Assisted Bi-Level Programming for Reward Learning from Internet Videoscited by 3
  4. 2023Toward Computationally Efficient Inverse Reinforcement Learning via Reward Shapingcited by 2
  5. 2024The State of Robot Motion Generationcited by 2
  6. 2024Automated Feature Selection for Inverse Reinforcement Learningcited by 1
  7. 2024RILe: Reinforced Imitation Learningcited by 1
  8. 2023On the Effective Horizon of Inverse Reinforcement Learningcited by 0
  9. 2024On Reward Transferability in Adversarial Inverse Reinforcement Learning: Insights from Random Matrix Theorycited by 0
  10. 2025Recursive Deep Inverse Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024The Virtues of Pessimism in Inverse Reinforcement Learning
  2. 2021A Generalised Inverse Reinforcement Learning Framework
  3. 2023Inverse Reinforcement Learning without Reinforcement Learning
  4. 2021Inverse Reinforcement Learning in a Continuous State Space with Formal Guarantees
  5. 2023Is Inverse Reinforcement Learning Harder than Standard Reinforcement Learning? A Theoretical Perspective
  6. 2024Hybrid Inverse Reinforcement Learning
  7. 2018A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progress
  8. 2025Inverse Reinforcement Learning via Convex Optimization