# Dynamic Measurement Scheduling for Event Forecasting using Deep RL

arXiv: https://arxiv.org/abs/1901.09699 · HTML: https://mlsanity.com/paper/1901.09699/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Jan 2019.

## Built on by (most cited first, 7 of 7)

- [Evaluation of Active Feature Acquisition Methods for Time-varying Feature Settings](https://mlsanity.com/paper/2312.01530/) (2023)
- [Data Minimization at Inference Time](https://mlsanity.com/paper/2305.17593/) (2023)
- [Evaluation of Active Feature Acquisition Methods for Static Feature Settings](https://mlsanity.com/paper/2312.03619/) (2023)
- [Measurement Scheduling for ICU Patients with Offline Reinforcement Learning](https://mlsanity.com/paper/2402.07344/) (2024)
- [Towards Cost Sensitive Decision Making](https://mlsanity.com/paper/2410.03892/) (2024)
- [ExOSITO: Explainable Off-Policy Learning with Side Information for Intensive Care Unit Blood Test Orders](https://mlsanity.com/paper/2504.17277/) (2025)
- [Extracting Expert's Goals by What-if Interpretable Modeling](https://mlsanity.com/paper/2110.15165/) (2021)

## Abstract (arXiv)

Imagine a patient in critical condition. What and when should be measured to forecast detrimental events, especially under the budget constraints? We answer this question by deep reinforcement learning (RL) that jointly minimizes the measurement cost and maximizes predictive gain, by scheduling strategically-timed measurements. We learn our policy to be dynamically dependent on the patient's health history. To scale our framework to exponentially large action space, we distribute our reward in a sequential setting that makes the learning easier. In our simulation, our policy outperforms heuristic-based scheduling with higher predictive gain and lower cost. In a real-world ICU mortality prediction task (MIMIC3), our policies reduce the total number of measurements by 31% or improve predictive gain by a factor of 3 as compared to physicians, under the off-policy policy evaluation.