mlsanityData to 7 Oct 2026

Distributional Inverse Reinforcement Learning

arXiv 2510.03013 ↗First posted 3 Oct 2025cs.LGcited by 0

On the map: Offline RL and imitation learning from data → Inverse reinforcement learning

Abstract (arXiv)

We propose a distributional framework for offline Inverse Reinforcement Learning (IRL) that jointly models uncertainty over reward functions and full distributions of returns. Unlike conventional IRL approaches that recover a deterministic reward estimate or match only expected returns, our method captures richer structure in expert behavior, particularly in learning the reward distribution, by minimizing first-order stochastic dominance (FSD) violations and thus integrating distortion risk measures (DRMs) into policy learning, enabling the recovery of both reward distributions and distribution-aware policies. This formulation is well-suited for behavior analysis and risk-aware imitation learning. Theoretical analysis shows that the algorithm converges with O(varepsilon-2) iteration complexity. Empirical results on synthetic benchmarks, real-world neurobehavioral data, and MuJoCo control tasks demonstrate that our method recovers expressive reward representations and achieves state-of-the-art performance.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 39 of its 75 references that are in the map. First posted on arXiv 3 Oct 2025.

Builds on

39 of 75 references.

Closest ancestors · 12

Same direction, newest first

  1. 2025Inverse Reinforcement Learning with Switching Rewards and History Dependency for Characterizing Animal Behaviors22 Jan
  2. 2024Enhancing Inverse Reinforcement Learning through Encoding Dynamic Information in Reward Shaping4 Oct
  3. 2024Walking the Values in Bayesian Inverse Reinforcement Learning15 Jul
  4. 2023A Bayesian Approach to Robust Inverse Reinforcement Learning15 Sep
  5. 2023Inverse Reinforcement Learning with the Average Reward Criterion24 May
  6. 2023When Demonstrations Meet Generative World Models: A Maximum Likelihood Framework for Offline Inverse Reinforcement Learning15 Feb
  7. 2022Maximum-Likelihood Inverse Reinforcement Learning with Finite-Time Guarantees4 Oct
  8. 2022A Primer on Maximum Causal Entropy Inverse Reinforcement Learning22 Mar
  9. 2021Scalable Bayesian Inverse Reinforcement Learning12 Feb
  10. 2020f-IRL: Inverse Reinforcement Learning via State Marginal Matching9 Nov
  11. 2020Robust Inverse Reinforcement Learning under Transition Dynamics Mismatch2 Jul
  12. 2018A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progress18 Jun

Other references · 27

  1. 2016Generative Adversarial Imitation Learningcited by 1,576
  2. 2016Variational Inference: A Review for Statisticianscited by 1,466
  3. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
  4. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,245
  5. 2017A Distributional Perspective on Reinforcement Learningcited by 743
  6. 2017Distributional Reinforcement Learning with Quantile Regressioncited by 464
  7. 2018Implicit Quantile Networks for Distributional Reinforcement Learningcited by 362
  8. 2023IDQL: Implicit Q-Learning as an Actor-Critic Method with Diffusion Policiescited by 307
  9. 2021IQ-Learn: Inverse soft-Q Learning for Imitationcited by 191
  10. 2019Imitation Learning via Off-Policy Distribution Matchingcited by 182
  11. 2015Maximum Entropy Deep Inverse Reinforcement Learningcited by 173
  12. 2020Distributional Soft Actor-Critic: Off-Policy Reinforcement Learning for Addressing Value Estimation Errorscited by 80
15 more
  1. 2021Risk-Averse Offline Reinforcement Learningcited by 65
  2. 2020Efficient Sampling-Based Maximum Entropy Inverse Reinforcement Learning with Application to Autonomous Drivingcited by 34
  3. 2019Driving with Style: Inverse Reinforcement Learning in General-Purpose Planning for Automated Drivingcited by 30
  4. 2018Risk-Sensitive Generative Adversarial Imitation Learningcited by 16
  5. 2024Foundations of Multivariate Distributional Reinforcement Learningcited by 16
  6. 2024Diffusing States and Matching Scores: A New Framework for Imitation Learningcited by 16
  7. 2023Internally Rewarded Reinforcement Learningcited by 14
  8. 2017Risk-sensitive Inverse Reinforcement Learning via Semi- and Non-Parametric Methodscited by 12
  9. 2023Learning Risk-Aware Quadrupedal Locomotion using Distributional Reinforcement Learningcited by 12
  10. 2023Infer and Adapt: Bipedal Locomotion Reward Learning from Demonstrations via Inverse Reinforcement Learningcited by 5
  11. 2024Bellman Diffusion: Generative Modeling as Learning a Linear Operator in the Distribution Spacecited by 4
  12. 2025From Demonstrations to Rewards: Alignment Without Explicit Human Preferencescited by 4
  13. 2023Eliciting Risk Aversion with Inverse Reinforcement Learning via Interactive Questioningcited by 2
  14. 2022Contractivity of Bellman Operator in Risk Averse Dynamic Programming with Infinite Horizoncited by 1
  15. 2025Imitation Learning as Return Distribution Matchingcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2021Versatile Inverse Reinforcement Learning via Cumulative Rewards
  2. 2021OPIRL: Sample Efficient Off-Policy Inverse Reinforcement Learning via Distribution Matching
  3. 2023Is Inverse Reinforcement Learning Harder than Standard Reinforcement Learning? A Theoretical Perspective
  4. 2026A Lecture Note on Offline RL and IRL, Part II: Foundations of Inverse Reinforcement Learning and Dynamic Discrete Choice Models
  5. 2023A Novel Variational Lower Bound for Inverse Reinforcement Learning
  6. 2024Diffusion Policies for Risk-Averse Behavior Modeling in Offline Reinforcement Learning
  7. 2020Efficient Exploration of Reward Functions in Inverse Reinforcement Learning via Bayesian Optimization
  8. 2026Inverse Reinforcement Learning without an Optimal Demonstrator: A Feasible Reward Set Approach