mlsanityData to 7 Oct 2026

Medical Dead-ends and Learning to Identify High-risk States and Treatments

arXiv 2110.04186 ↗First posted 8 Oct 2021cs.LG cs.AIcited by 28

Not on the map. Why?

Abstract (arXiv)

Machine learning has successfully framed many sequential decision making problems as either supervised prediction, or optimal decision-making policy identification via reinforcement learning. In data-constrained offline settings, both approaches may fail as they assume fully optimal behavior or rely on exploring alternatives that may not exist. We introduce an inherently different approach that identifies possible "dead-ends" of a state space. We focus on the condition of patients in the intensive care unit, where a "medical dead-end" indicates that a patient will expire, regardless of all potential future treatment sequences. We postulate "treatment security" as avoiding treatments with probability proportional to their chance of leading to dead-ends, present a formal proof, and frame discovery as an RL problem. We then train three independent deep neural models for automated state construction, dead-end discovery and confirmation. Our empirical results discover that dead-ends exist in real clinical data among septic patients, and further reveal gaps between secure treatments and those that were administered.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). Builds on 23 of its 65 references that are in the map. First posted on arXiv 8 Oct 2021.

Builds on

23 of 65 references.

References · 23

  1. 2015Continuous control with deep reinforcement learningcited by 3,279
  2. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  3. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,704
  4. 2014On the Properties of Neural Machine Translation: Encoder-Decoder Approachescited by 1,303
  5. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,227
  6. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 866
  7. 2019Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reductioncited by 833
  8. 2019Behavior Regularized Offline Reinforcement Learningcited by 654
  9. 2019Reinforcement Learning in Healthcare: A Surveycited by 308
  10. 2016Cooperative Inverse Reinforcement Learningcited by 298
  11. 2019Way Off-Policy Batch Deep Reinforcement Learning of Implicit Human Preferences in Dialogcited by 283
  12. 2020Critic Regularized Regressioncited by 269
11 more
  1. 2017Inverse Reward Designcited by 246
  2. 2017Safe Policy Improvement with Baseline Bootstrappingcited by 141
  3. 2021S4RL: Surprisingly Simple Self-Supervision for Offline Reinforcement Learningcited by 85
  4. 2017Continuous State-Space Models for Optimal Sepsis Treatment - a Deep Reinforcement Learning Approachcited by 68
  5. 2019Improving Sepsis Treatment Strategies by Combining Deep and Kernel-Based Reinforcement Learningcited by 34
  6. 2020An Empirical Study of Representation Learning for Reinforcement Learning in Healthcarecited by 34
  7. 2019Off-Policy Evaluation via Off-Policy Classificationcited by 32
  8. 2020Clinician-in-the-Loop Decision Making: Reinforcement Learning with Near-Optimal Set-Valued Policiescited by 24
  9. 2017An Improved Multi-Output Gaussian Process RNN with Real-Time Validation for Early Sepsis Detectioncited by 16
  10. 2017On overfitting and asymptotic bias in batch reinforcement learning with partial observabilitycited by 16
  11. 2019Optimizing Sequential Medical Treatments with Auto-Encoding Heuristic Search in POMDPscited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 3; 2023: 14; 2024: 4; 2025: 4; 2026: 2.

28 papers.

Sort:

By area:

  1. 2025Concise Reasoning via Reinforcement Learningcited by 78
  2. 2023The Curious Price of Distributional Robustness in Reinforcement Learning with a Generative Modelcited by 50
  3. 2023Leveraging Factored Action Spaces for Efficient Offline Reinforcement Learning in Healthcarecited by 41
  4. 2022Provably Efficient Risk-Sensitive Reinforcement Learning: Iterated CVaR and Worst Pathcited by 21
  5. 2022Semi-Markov Offline Reinforcement Learning for Healthcarecited by 18
  6. 2023Deep Offline Reinforcement Learning for Real-world Treatment Optimization Applicationscited by 17
  7. 2023Systematic Rectification of Language Models via Dead-end Analysiscited by 16
  8. 2022Learning General World Models in a Handful of Reward-Free Deploymentscited by 12
  9. 2023Quasi-optimal Reinforcement Learning with Continuous Actionscited by 7
  10. 2023Connected and Automated Vehicles in Mixed-Traffic: Learning Human Driver Behavior for Effective On-Ramp Mergingcited by 7
  11. 2024TrajDeleter: Enabling Trajectory Forgetting in Offline Reinforcement Learning Agentscited by 7
  12. 2023Risk Sensitive Dead-end Identification in Safety-Critical Offline Reinforcement Learningcited by 6
16 more
  1. 2025The Courage to Stop: Overcoming Sunk Cost Fallacy in Deep Reinforcement Learningcited by 6
  2. 2023Minimum Levels of Interpretability for Artificial Moral Agentscited by 5
  3. 2023Safe Reinforcement Learning with Dead-Ends Avoidance and Recoverycited by 4
  4. 2024Offline Inverse Constrained Reinforcement Learning for Safe-Critical Decision Making in Healthcarecited by 4
  5. 2025Exploring Time-Step Size in Reinforcement Learning for Sepsis Treatmentcited by 4
  6. 2023Approximate Information States for Worst-Case Control and Learning in Uncertain Systemscited by 3
  7. 2023Pruning the Way to Reliable Policies: A Multi-Objective Deep Q-Learning Approach to Critical Carecited by 2
  8. 2023Successor Features for Efficient Multisubject Controlled Text Generationcited by 2
  9. 2025PrivORL: Differentially Private Synthetic Dataset for Offline Reinforcement Learningcited by 1
  10. 2026The hidden risks of temporal resampling in clinical reinforcement learningcited by 1
  11. 2021Extracting Expert's Goals by What-if Interpretable Modelingcited by 0
  12. 2023Learned Risk Metric Maps for Kinodynamic Systemscited by 0
  13. 2023Learning and DiSentangling Patient Static Information from Time-series Electronic HEalth Record (STEER)cited by 0
  14. 2024A Dynamical View of the Question of Whycited by 0
  15. 2024Bayesian Inverse Transition Learning: Learning Dynamics From Near-Optimal Trajectoriescited by 0
  16. 2026Insulin4RL: Real-Time Insulin Management in the Intensive Care Unit for Offline Reinforcement Learningcited by 0