# Looking for Out-of-Distribution Environments in Multi-center Critical Care Data

arXiv: https://arxiv.org/abs/2205.13398 · HTML: https://mlsanity.com/paper/2205.13398/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 26 references that are in the map.
- First posted on arXiv 26 May 2022.

## Builds on (most cited first, 14 of 14)

- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [WILDS: A Benchmark of in-the-Wild Distribution Shifts](https://mlsanity.com/paper/2012.07421/) (2020)
- [In Search of Lost Domain Generalization](https://mlsanity.com/paper/2007.01434/) (2020)
- [Learning to Generalize: Meta-Learning for Domain Generalization](https://mlsanity.com/paper/1710.03463/) (2017)
- [Recognition in Terra Incognita](https://mlsanity.com/paper/1807.04975/) (2018)
- [Causality matters in medical imaging](https://mlsanity.com/paper/1912.08142/) (2019)
- [Dark Model Adaptation: Semantic Image Segmentation from Daytime to Nighttime](https://mlsanity.com/paper/1810.02575/) (2018)
- [Feature Robustness in Non-stationary Health Records: Caveats to Deployable Model Performance in Common Clinical Machine Learning Tasks](https://mlsanity.com/paper/1908.00690/) (2019)
- [Benchmarking machine learning models on multi-centre eICU critical care dataset](https://mlsanity.com/paper/1910.00964/) (2019)
- [An Empirical Framework for Domain Generalization in Clinical Settings](https://mlsanity.com/paper/2103.11163/) (2021)
- [Out-of-Distribution Detection for Medical Applications: Guidelines for Practical Evaluation](https://mlsanity.com/paper/2109.14885/) (2021)
- [Temporal Pointwise Convolutional Networks for Length of Stay Prediction in the Intensive Care Unit](https://mlsanity.com/paper/2007.09483/) (2020)
- [Generalizability of predictive models for intensive care unit patients](https://mlsanity.com/paper/1812.02275/) (2018)

## Built on by (most cited first, 3 of 3)

- [The Data Addition Dilemma](https://mlsanity.com/paper/2408.04154/) (2024)
- [From Single-Hospital to Multi-Centre Applications: Enhancing the Generalisability of Deep Learning Models for Adverse Event Prediction in the ICU](https://mlsanity.com/paper/2303.15354/) (2023)
- [Prediction of Survival Outcomes under Clinical Presence Shift: A Joint Neural Network Architecture](https://mlsanity.com/paper/2508.05472/) (2025)

## Abstract (arXiv)

Clinical machine learning models show a significant performance drop when tested in settings not seen during training. Domain generalisation models promise to alleviate this problem, however, there is still scepticism about whether they improve over traditional training. In this work, we take a principled approach to identifying Out of Distribution (OoD) environments, motivated by the problem of cross-hospital generalization in critical care. We propose model-based and heuristic approaches to identify OoD environments and systematically compare models with different levels of held-out information. We find that access to OoD data does not translate to increased performance, pointing to inherent limitations in defining potential OoD environments potentially due to data harmonisation and sampling. Echoing similar results with other popular clinical benchmarks in the literature, new approaches are required to evaluate robust models on health records.