mlsanityData to 7 Oct 2026

DO-IQS: Dynamics-Aware Offline Inverse Q-Learning for Optimal Stopping with Unknown Gain Functions

arXiv 2503.03515 ↗First posted 5 Mar 2025stat.ML cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

We consider the Inverse Optimal Stopping (IOS) problem where, based on stopped expert trajectories, one aims to recover the optimal stopping region through the continuation and stopping gain functions approximation. The uniqueness of the stopping region allows the use of IOS in real-world applications with safety concerns. Although current state-of-the-art inverse reinforcement learning methods recover both a Q-function and the corresponding optimal policy, they fail to account for specific challenges posed by optimal stopping problems. These include data sparsity near the stopping region, the non-Markovian nature of the continuation gain, a proper treatment of boundary conditions, the need for a stable offline approach for risk-sensitive applications, and a lack of a quality evaluation metric. These challenges are addressed with the proposed Dynamics-Aware Offline Inverse Q-Learning for Optimal Stopping (DO-IQS), which incorporates temporal information by approximating the cumulative continuation gain together with the world dynamics and the Q-function without querying to the environment. In addition, a confidence-based oversampling approach is proposed to treat the data sparsity problem. We demonstrate the performance of our models on real and artificial data including an optimal intervention for the critical events problem.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 15 of its 39 references that are in the map. First posted on arXiv 5 Mar 2025.

Builds on

15 of 39 references.

References · 15

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  2. 2016Generative Adversarial Imitation Learningcited by 1,576
  3. 2018Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learningcited by 203
  4. 2021IQ-Learn: Inverse soft-Q Learning for Imitationcited by 191
  5. 2019Imitation Learning from Imperfect Demonstrationcited by 112
  6. 2022Discriminator-Weighted Offline Imitation Learning from Suboptimal Demonstrationscited by 89
  7. 2019Generative Adversarial Minority Oversamplingcited by 72
  8. 2023LS-IQ: Implicit Reward Regularization for Inverse Reinforcement Learningcited by 28
  9. 2023When Demonstrations Meet Generative World Models: A Maximum Likelihood Framework for Offline Inverse Reinforcement Learningcited by 26
  10. 2016Inverse Reinforcement Learning with Simultaneous Estimation of Rewards and Dynamicscited by 24
  11. 2023CLARE: Conservative Model-Based Reward Learning for Offline Inverse Reinforcement Learningcited by 20
  12. 2020Necessary and Sufficient Conditions for Inverse Reinforcement Learning of Bayesian Stopping Time Problemscited by 2
3 more
  1. 2013Behavior Pattern Recognition using A New Representation Modelcited by 1
  2. 2014An inverse optimal stopping problem for diffusion processescited by 1
  3. 2022A Class of Mean-Field Games with Optimal Stopping and its Inverse Problemcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Offline Reinforcement Learning via Inverse Optimization
  2. 2025BiCQL-ML: A Bi-Level Conservative Q-Learning Framework for Maximum Likelihood Inverse Reinforcement Learning
  3. 2026Residuals-based Offline Reinforcement Learning
  4. 2026Loop-Free Inverse Reinforcement Learning via Sequential Value Recovery with Q-Score Matching
  5. 2023Contextual Conservative Q-Learning for Offline Reinforcement Learning
  6. 2020Deep Inverse Q-learning with Constraints
  7. 2023Bayesian Inverse Transition Learning for Offline Settings
  8. 2021A Generalised Inverse Reinforcement Learning Framework