# Robust Real-Time Mortality Prediction in the Intensive Care Unit using Temporal Difference Learning

arXiv: https://arxiv.org/abs/2411.04285 · HTML: https://mlsanity.com/paper/2411.04285/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 29 references that are in the map.
- First posted on arXiv 6 Nov 2024.

## Builds on (most cited first, 6 of 6)

- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems](https://mlsanity.com/paper/2005.01643/) (2020)
- [Self-Supervised Transformer for Sparse and Irregularly Sampled Multivariate Clinical Time-Series](https://mlsanity.com/paper/2107.14293/) (2021)
- [A Reinforcement Learning Approach to Weaning of Mechanical Ventilation in Intensive Care Units](https://mlsanity.com/paper/1704.06300/) (2017)
- [How to set AdamW's weight decay as you scale model and dataset size](https://mlsanity.com/paper/2405.13698/) (2024)
- [Predicting the Need for Blood Transfusion in Intensive Care Units with Reinforcement Learning](https://mlsanity.com/paper/2206.14198/) (2022)

## Built on by (most cited first, 1 of 1)

- [Early Prediction of In-Hospital ICU Mortality Using Innovative First-Day Data: A Review](https://mlsanity.com/paper/2505.12344/) (2025)

## Abstract (arXiv)

The task of predicting long-term patient outcomes using supervised machine learning is a challenging one, in part because of the high variance of each patient's trajectory, which can result in the model over-fitting to the training data. Temporal difference (TD) learning, a common reinforcement learning technique, may reduce variance by generalising learning to the pattern of state transitions rather than terminal outcomes. However, in healthcare this method requires several strong assumptions about patient states, and there appears to be limited literature evaluating the performance of TD learning against traditional supervised learning methods for long-term health outcome prediction tasks. In this study, we define a framework for applying TD learning to real-time irregularly sampled time series data using a Semi-Markov Reward Process. We evaluate the model framework in predicting intensive care mortality and show that TD learning under this framework can result in improved model robustness compared to standard supervised learning methods. and that this robustness is maintained even when validated on external datasets. This approach may offer a more reliable method when learning to predict patient outcomes using high-variance irregular time series data.