mlsanityData to 7 Oct 2026

Versatile Inverse Reinforcement Learning via Cumulative Rewards

arXiv 2111.07667 ↗First posted 15 Nov 2021cs.LGcited by 2

On the map: Offline RL and imitation learning from data → Inverse reinforcement learning

Abstract (arXiv)

Inverse Reinforcement Learning infers a reward function from expert demonstrations, aiming to encode the behavior and intentions of the expert. Current approaches usually do this with generative and uni-modal models, meaning that they encode a single behavior. In the common setting, where there are various solutions to a problem and the experts show versatile behavior this severely limits the generalization capabilities of these methods. We propose a novel method for Inverse Reinforcement Learning that overcomes these problems by formulating the recovered reward as a sum of iteratively trained discriminators. We show on simulated tasks that our approach is able to recover general, high-quality reward functions and produces policies of the same quality as behavioral cloning approaches designed for versatile behavior.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 14 of its 33 references that are in the map. First posted on arXiv 15 Nov 2021.

Builds on

14 of 33 references.

Closest ancestors · 1

Same direction, newest first

  1. 2019Extrapolating Beyond Suboptimal Demonstrations via Inverse Reinforcement Learning from Observations12 Apr

Other references · 13

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  3. 2015Trust Region Policy Optimizationcited by 2,503
  4. 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,455
  5. 2016Generative Adversarial Imitation Learningcited by 1,576
  6. 2016Variational Inference: A Review for Statisticianscited by 1,466
  7. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,084
  8. 2017Reinforcement Learning with Deep Energy-Based Policiescited by 659
  9. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 400
  10. 2019A Divergence Minimization Perspective on Imitation Learning Methodscited by 188
  11. 2020Non-Adversarial Imitation Learning and its Connections to Adversarial Methodscited by 16
  12. 2019Trust-Region Variational Inference with Gaussian Mixture Modelscited by 14
1 more
  1. 2020Expected Information Maximization: Using the I-Projection for Mixture Density Estimationcited by 9

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 1.

2 papers.

  1. 2022Inferring Versatile Behavior from Demonstrations by Matching Geometric Descriptorscited by 6
  2. 2023Curriculum-Based Imitation of Versatile Skillscited by 3

Similar, not cited

Close in wording, no citation either way

  1. 2025Generalizing Behavior via Inverse Reinforcement Learning with Closed-Form Reward Centroids
  2. 2024Multi Task Inverse Reinforcement Learning for Common Sense Reward
  3. 2020Inverse Reinforcement Learning from a Gradient-based Learner
  4. 2023BC-IRL: Learning Generalizable Reward Functions from Demonstrations
  5. 2019Meta-Inverse Reinforcement Learning with Probabilistic Context Variables
  6. 2024Learning Causally Invariant Reward Functions from Diverse Demonstrations
  7. 2018Learning a Prior over Intent via Meta-Inverse Reinforcement Learning
  8. 2026Loop-Free Inverse Reinforcement Learning via Sequential Value Recovery with Q-Score Matching