mlsanityData to 7 Oct 2026

Learning Time-Invariant Reward Functions through Model-Based Inverse Reinforcement Learning

arXiv 2107.03186 ↗First posted 7 Jul 2021cs.RO stat.MLcited by 0

On the map: Offline RL and imitation learning from data → Inverse reinforcement learning

Abstract (arXiv)

Inverse reinforcement learning is a paradigm motivated by the goal of learning general reward functions from demonstrated behaviours. Yet the notion of generality for learnt costs is often evaluated in terms of robustness to various spatial perturbations only, assuming deployment at fixed speeds of execution. However, this is impractical in the context of robotics and building, time-invariant solutions is of crucial importance. In this work, we propose a formulation that allows us to 1) vary the length of execution by learning time-invariant costs, and 2) relax the temporal alignment requirements for learning from demonstration. We apply our method to two different types of cost formulations and evaluate their performance in the context of learning reward functions for simulated placement and peg in hole tasks executed on a 7DoF Kuka IIWA arm. Our results show that our approach enables learning temporally invariant rewards from misaligned demonstration that can also generalise spatially to out of distribution tasks.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 12 of its 29 references that are in the map. First posted on arXiv 7 Jul 2021.

Builds on

12 of 29 references.

References · 12

  1. 2016Generative Adversarial Imitation Learningcited by 1,576
  2. 2020robosuite: A Modular Simulation Framework and Benchmark for Robot Learningcited by 484
  3. 2018An Algorithmic Perspective on Imitation Learningcited by 455
  4. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 400
  5. 2019Generalized Inner Loop Meta-Learningcited by 92
  6. 2019State Alignment-based Imitation Learningcited by 73
  7. 2019Meta-Learning via Learned Losscited by 60
  8. 2020Model-Based Inverse Reinforcement Learning from Visual Demonstrationscited by 52
  9. 2020Encoding Physical Constraints in Differentiable Newton-Euler Algorithmcited by 23
  10. 2019Adversarial Imitation Learning from Incomplete Demonstrationscited by 16
  11. 2020Learning from Sparse Demonstrationscited by 9
  12. 2020Learning from Demonstration with Weakly Supervised Disentanglementcited by 8

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2020Efficient Exploration of Reward Functions in Inverse Reinforcement Learning via Bayesian Optimization
  2. 2019Meta-Inverse Reinforcement Learning with Probabilistic Context Variables
  3. 2025Cost Function Estimation Using Inverse Reinforcement Learning with Minimal Observations
  4. 2018Learning a Prior over Intent via Meta-Inverse Reinforcement Learning
  5. 2024Learning Causally Invariant Reward Functions from Diverse Demonstrations
  6. 2023BC-IRL: Learning Generalizable Reward Functions from Demonstrations
  7. 2019Option Compatible Reward Inverse Reinforcement Learning
  8. 2020Active Task-Inference-Guided Deep Inverse Reinforcement Learning