# Measurement Scheduling for ICU Patients with Offline Reinforcement Learning

arXiv: https://arxiv.org/abs/2402.07344 · HTML: https://mlsanity.com/paper/2402.07344/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Safe offline RL, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 8 of its 21 references that are in the map.
- First posted on arXiv 12 Feb 2024.

## Placed in

- Offline RL and imitation learning from data / [Safe offline RL](https://mlsanity.com/offlinerl/safe-offline-rl/)

## Builds on (most cited first, 8 of 8)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Conservative Q-Learning for Offline Reinforcement Learning](https://mlsanity.com/paper/2006.04779/) (2020)
- [Offline Reinforcement Learning with Implicit Q-Learning](https://mlsanity.com/paper/2110.06169/) (2021)
- [Doubly Robust Off-policy Value Evaluation for Reinforcement Learning](https://mlsanity.com/paper/1511.03722/) (2015)
- [Continuous State-Space Models for Optimal Sepsis Treatment - a Deep Reinforcement Learning Approach](https://mlsanity.com/paper/1705.08422/) (2017)
- [A Reinforcement Learning Approach to Weaning of Mechanical Ventilation in Intensive Care Units](https://mlsanity.com/paper/1704.06300/) (2017)
- [An Optimal Policy for Patient Laboratory Tests in Intensive Care Units](https://mlsanity.com/paper/1808.04679/) (2018)
- [Dynamic Measurement Scheduling for Event Forecasting using Deep RL](https://mlsanity.com/paper/1901.09699/) (2019)

## Built on by (most cited first, 3 of 3)

- [The Curious Language Model: Strategic Test-Time Information Acquisition](https://mlsanity.com/paper/2506.09173/) (2025)
- [Can we generate portable representations for clinical time series data using LLMs?](https://mlsanity.com/paper/2603.23987/) (2026)
- [ExOSITO: Explainable Off-Policy Learning with Side Information for Intensive Care Unit Blood Test Orders](https://mlsanity.com/paper/2504.17277/) (2025)

## Abstract (arXiv)

Scheduling laboratory tests for ICU patients presents a significant challenge. Studies show that 20-40% of lab tests ordered in the ICU are redundant and could be eliminated without compromising patient safety. Prior work has leveraged offline reinforcement learning (Offline-RL) to find optimal policies for ordering lab tests based on patient information. However, new ICU patient datasets have since been released, and various advancements have been made in Offline-RL methods. In this study, we first introduce a preprocessing pipeline for the newly-released MIMIC-IV dataset geared toward time-series tasks. We then explore the efficacy of state-of-the-art Offline-RL methods in identifying better policies for ICU patient lab test scheduling. Besides assessing methodological performance, we also discuss the overall suitability and practicality of using Offline-RL frameworks for scheduling laboratory tests in ICU settings.