mlsanityData to 14 Oct 2026

Reward Transfer from Inverse Reinforcement Learning: A Coupled Minimax Approach

arXiv 2605.27834 ↗First posted 27 May 2026cs.LG stat.MLcited by 1

On the map: Offline RL and imitation learning from data → Inverse reinforcement learning

also in 1 more
Abstract (arXiv)

We study the transfer of rewards learned using inverse reinforcement learning from expert demonstrations in one environment to reinforcement learning in a new, different environment. This arises naturally when demonstrations are collected in a controlled environment. We formulate the problem as a joint system of Bellman equations across the source and target environments and develop minimax estimators for the target soft-q-function. Whereas a sequential solution approach first estimates the source reward and then plugs it into the target control problem, a coupled approach solves the source and target system of equations jointly. We show that, in contrast to the sequential approach, the coupled approach removes the first-order influence of source Bellman residual error. We characterize the local behavior of each approach, develop finite-sample soft-q-function error bounds, and prove regret guarantees for the resulting soft-control policy. An empirical investigation using a sepsis simulator validates the theoretical comparison.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 4 of its 43 references that are in the map. First posted on arXiv 27 May 2026.

Builds on

4 of 43 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Efficient Inference for Inverse Reinforcement Learning and Dynamic Discrete Choice Models30 Dec

Other references · 3

  1. 2018Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Reviewcited by 482
  2. 2017On the Properties of the Softmax Function with Application in Game Theory and Reinforcement Learningcited by 142
  3. 2025Debiased Inference for High-Dimensional Regression Models Based on Profile M-Estimationcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026PHIRL: Aligning Learned Rewards with Task Progress for Inverse Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Loop-Free Inverse Reinforcement Learning via Sequential Value Recovery with Q-Score Matching
  2. 2019Option Compatible Reward Inverse Reinforcement Learning
  3. 2025Inverse Reinforcement Learning with Just Classification and a Few Regressions
  4. 2026Inverse Reinforcement Learning without an Optimal Demonstrator: A Feasible Reward Set Approach
  5. 2019Meta-Inverse Reinforcement Learning with Probabilistic Context Variables
  6. 2021Versatile Inverse Reinforcement Learning via Cumulative Rewards
  7. 2023Is Inverse Reinforcement Learning Harder than Standard Reinforcement Learning? A Theoretical Perspective
  8. 2024Learning Causally Invariant Reward Functions from Diverse Demonstrations