mlsanityData to 7 Oct 2026

Bayesian Inverse Transition Learning: Learning Dynamics From Near-Optimal Trajectories

arXiv 2411.05174 ↗First posted 7 Nov 2024cs.LG cs.AI stat.MLcited by 0

Not on the map. Why?

Abstract (arXiv)

We consider the problem of estimating the transition dynamics T^* from near-optimal expert trajectories in the context of offline model-based reinforcement learning. We develop a novel constraint-based method, Inverse Transition Learning, that treats the limited coverage of the expert trajectories as a feature: we use the fact that the expert is near-optimal to inform our estimate of T^*. We integrate our constraints into a Bayesian approach. Across both synthetic environments and real healthcare scenarios like Intensive Care Unit (ICU) patient management in hypotension, we demonstrate not only significant improvements in decision-making, but that our posterior can inform when transfer will be successful.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 19 of its 59 references that are in the map. First posted on arXiv 7 Nov 2024.

Builds on

19 of 59 references.

References · 19

  1. 2018Recurrent World Models Facilitate Policy Evolutioncited by 761
  2. 2016CVXPY: A Python-Embedded Modeling Language for Convex Optimizationcited by 633
  3. 2020MOReL : Model-Based Offline Reinforcement Learningcited by 582
  4. 2020Learning Invariant Representations for Reinforcement Learning without Reconstructioncited by 424
  5. 2021COMBO: Conservative Offline Model-Based Policy Optimizationcited by 365
  6. 2019When to use parametric models in reinforcement learning?cited by 118
  7. 2020Invariant Causal Prediction for Block MDPscited by 104
  8. 2018Woulda, Coulda, Shoulda: Counterfactually-Guided Policy Searchcited by 81
  9. 2023Leveraging Factored Action Spaces for Efficient Offline Reinforcement Learning in Healthcarecited by 41
  10. 2018Where Do You Think You're Going?: Inferring Beliefs about Dynamics from Behaviorcited by 40
  11. 2019Maximum Likelihood Constraint Inference for Inverse Reinforcement Learningcited by 33
  12. 2021Medical Dead-ends and Learning to Identify High-risk States and Treatmentscited by 28
7 more
  1. 2016Inverse Reinforcement Learning with Simultaneous Estimation of Rewards and Dynamicscited by 24
  2. 2020Clinician-in-the-Loop Decision Making: Reinforcement Learning with Near-Optimal Set-Valued Policiescited by 24
  3. 2022Model-Based Offline Reinforcement Learning with Pessimism-Modulated Dynamics Beliefcited by 22
  4. 2013Model-Based Bayesian Explorationcited by 21
  5. 2023Model-based Offline Reinforcement Learning with Count-based Conservatismcited by 16
  6. 2019Learning and Planning for Time-Varying MDPs Using Maximum Likelihood Estimationcited by 10
  7. 2024ICU-Sepsis: A Benchmark MDP Built from Real Medical Datacited by 6

Built on by

Not cited yet.