# Towards Safe Mechanical Ventilation Treatment Using Deep Offline Reinforcement Learning

arXiv: https://arxiv.org/abs/2210.02552 · HTML: https://mlsanity.com/paper/2210.02552/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Placed in Safe offline RL, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 10 of its 32 references that are in the map.
- First posted on arXiv 5 Oct 2022.

## Placed in

- Offline RL and imitation learning from data / [Safe offline RL](https://mlsanity.com/offlinerl/safe-offline-rl/)

## Builds on (most cited first, 10 of 10)

- [Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems](https://mlsanity.com/paper/2005.01643/) (2020)
- [Conservative Q-Learning for Offline Reinforcement Learning](https://mlsanity.com/paper/2006.04779/) (2020)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Reinforcement Learning in Healthcare: A Survey](https://mlsanity.com/paper/1908.08796/) (2019)
- [Batch Policy Learning under Constraints](https://mlsanity.com/paper/1903.08738/) (2019)
- [Deep Reinforcement Learning for Sepsis Treatment](https://mlsanity.com/paper/1711.09602/) (2017)
- [Model Selection for Offline Reinforcement Learning: Practical Considerations for Healthcare Settings](https://mlsanity.com/paper/2107.11003/) (2021)
- [A Reinforcement Learning Approach to Weaning of Mechanical Ventilation in Intensive Care Units](https://mlsanity.com/paper/1704.06300/) (2017)
- [Improving Sepsis Treatment Strategies by Combining Deep and Kernel-Based Reinforcement Learning](https://mlsanity.com/paper/1901.04670/) (2019)
- [Machine Learning for Mechanical Ventilation Control](https://mlsanity.com/paper/2102.06779/) (2021)

## Built on by (most cited first, 16 of 16)

- [Temporal Logic Specification-Conditioned Decision Transformer for Offline Safe Reinforcement Learning](https://mlsanity.com/paper/2402.17217/) (2024)
- [medDreamer: Model-Based Reinforcement Learning with Latent Imagination on Complex EHRs for Clinical Decision Support](https://mlsanity.com/paper/2505.19785/) (2025)
- [Reinforcement Learning in Dynamic Treatment Regimes Needs Critical Reexamination](https://mlsanity.com/paper/2405.18556/) (2024)
- [Offline Inverse Constrained Reinforcement Learning for Safe-Critical Decision Making in Healthcare](https://mlsanity.com/paper/2410.07525/) (2024)
- [Beyond Prediction: Reinforcement Learning as the Defining Leap in Healthcare AI](https://mlsanity.com/paper/2508.21101/) (2025)
- [Towards Real-World Applications of Personalized Anesthesia Using Policy Constraint Q Learning for Propofol Infusion Control](https://mlsanity.com/paper/2303.10180/) (2023)
- [The hidden risks of temporal resampling in clinical reinforcement learning](https://mlsanity.com/paper/2602.06603/) (2026)
- [Safe and Interpretable Estimation of Optimal Treatment Regimes](https://mlsanity.com/paper/2310.15333/) (2023)
- [Methodology for Interpretable Reinforcement Learning for Optimizing Mechanical Ventilation](https://mlsanity.com/paper/2404.03105/) (2024)
- [Unmasking Societal Biases in Respiratory Support for ICU Patients through Social Determinants of Health](https://mlsanity.com/paper/2502.16477/) (2025)
- [ExOSITO: Explainable Off-Policy Learning with Side Information for Intensive Care Unit Blood Test Orders](https://mlsanity.com/paper/2504.17277/) (2025)
- [The Safety Challenges of Deep Learning in Real-World Type 1 Diabetes Management](https://mlsanity.com/paper/2310.14743/) (2023)
- [Inferring Relative Consequences of Mechanical Ventilation from Observational Data Using Game-Based Comparisons](https://mlsanity.com/paper/2510.15127/) (2025)
- [Ensuring Safety in Automated Mechanical Ventilation through Offline Reinforcement Learning and Digital Twin Verification](https://mlsanity.com/paper/2603.11372/) (2026)
- [VentAgent: When LLMs Learn to Breathe -- Multi-Objective Arbitration for ARDS Ventilation](https://mlsanity.com/paper/2606.04632/) (2026)
- [Evaluating covariate balance for long time horizon Markov decision processes](https://mlsanity.com/paper/2607.15080/) (2026)

## Abstract (arXiv)

Mechanical ventilation is a key form of life support for patients with pulmonary impairment. Healthcare workers are required to continuously adjust ventilator settings for each patient, a challenging and time consuming task. Hence, it would be beneficial to develop an automated decision support tool to optimize ventilation treatment. We present DeepVent, a Conservative Q-Learning (CQL) based offline Deep Reinforcement Learning (DRL) agent that learns to predict the optimal ventilator parameters for a patient to promote 90 day survival. We design a clinically relevant intermediate reward that encourages continuous improvement of the patient vitals as well as addresses the challenge of sparse reward in RL. We find that DeepVent recommends ventilation parameters within safe ranges, as outlined in recent clinical trials. The CQL algorithm offers additional safety by mitigating the overestimation of the value estimates of out-of-distribution states/actions. We evaluate our agent using Fitted Q Evaluation (FQE) and demonstrate that it outperforms physicians from the MIMIC-III dataset.