# HOLMES: Health OnLine Model Ensemble Serving for Deep Learning Models in Intensive Care Units

arXiv: https://arxiv.org/abs/2008.04063 · HTML: https://mlsanity.com/paper/2008.04063/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Aug 2020.

## Built on by (most cited first, 30 of 45)

- [Frozen Language Model Helps ECG Zero-Shot Learning](https://mlsanity.com/paper/2303.12311/) (2023)
- [NAS-Bench-360: Benchmarking Neural Architecture Search on Diverse Tasks](https://mlsanity.com/paper/2110.05668/) (2021)
- [An Electrocardiogram Foundation Model Built on over 10 Million Recordings with External Evaluation across Multiple Domains](https://mlsanity.com/paper/2410.04133/) (2024)
- [Efficient Architecture Search for Diverse Tasks](https://mlsanity.com/paper/2204.07554/) (2022)
- [DiffuSETS: 12-lead ECG Generation Conditioned on Clinical Text Reports and Patient-Specific Information](https://mlsanity.com/paper/2501.05932/) (2025)
- [WildPPG: A Real-World PPG Dataset of Long Continuous Recordings](https://mlsanity.com/paper/2412.17540/) (2024)
- [Can Brain Signals Reveal Inner Alignment with Human Languages?](https://mlsanity.com/paper/2208.06348/) (2022)
- [A Simple Self-Supervised ECG Representation Learning Method via Manipulated Temporal-Spatial Reverse Detection](https://mlsanity.com/paper/2202.12458/) (2022)
- [PPGFlowECG: Latent Rectified Flow with Cross-Modal Encoding for PPG-Guided ECG Generation and Cardiovascular Disease Detection](https://mlsanity.com/paper/2509.19774/) (2025)
- [CLEF: Clinically-Guided Contrastive Learning for Electrocardiogram Foundation Models](https://mlsanity.com/paper/2512.02180/) (2025)
- [Shifting the Paradigm: A Diffeomorphism Between Time Series Data Manifolds for Achieving Shift-Invariancy in Deep Learning](https://mlsanity.com/paper/2502.19921/) (2025)
- [Visually-aware Acoustic Event Detection using Heterogeneous Graphs](https://mlsanity.com/paper/2207.07935/) (2022)
- [Hierarchical Pruning of Deep Ensembles with Focal Diversity](https://mlsanity.com/paper/2311.10293/) (2023)
- [TusoAI: Agentic Optimization for Scientific Methods](https://mlsanity.com/paper/2509.23986/) (2025)
- [SupReMix: Supervised Contrastive Learning for Medical Imaging Regression with Mixup](https://mlsanity.com/paper/2309.16633/) (2023)
- [Deep Learning for Detecting and Early Predicting Chronic Obstructive Pulmonary Disease from Spirogram Time Series](https://mlsanity.com/paper/2405.03239/) (2024)
- [Dist Loss: Enhancing Regression in Few-Shot Region through Distribution Distance Constraint](https://mlsanity.com/paper/2411.15216/) (2024)
- [SHIELD: A Secure and Highly Enhanced Integrated Learning for Robust Deepfake Detection against Adversarial Attacks](https://mlsanity.com/paper/2507.13170/) (2025)
- [Spectral Propagation Graph Network for Few-shot Time Series Classification](https://mlsanity.com/paper/2202.04769/) (2022)
- [MetaVA: Curriculum Meta-learning and Pre-fine-tuning of Deep Neural Networks for Detecting Ventricular Arrhythmias based on ECGs](https://mlsanity.com/paper/2202.12450/) (2022)
- [Defending Against Adversarial Attack in ECG Classification with Adversarial Distillation Training](https://mlsanity.com/paper/2203.09487/) (2022)
- [Finding Short Signals in Long Irregular Time Series with Continuous-Time Attention Policy Networks](https://mlsanity.com/paper/2302.04052/) (2023)
- [How Much Is Hidden in the NAS Benchmarks? Few-Shot Adaptation of a NAS Predictor](https://mlsanity.com/paper/2311.18451/) (2023)
- [Knowledge Knows, Verbalization Tells: Disentangling Latent Directions for Mathematical Solvability in LLMs](https://mlsanity.com/paper/2607.05013/) (2026)
- [Subgraph Stationary Hardware-Software Inference Co-Design](https://mlsanity.com/paper/2306.17266/) (2023)
- [Deep Learning with Information Fusion and Model Interpretation for Health Monitoring of Fetus based on Long-term Prenatal Electronic Fetal Heart Rate Monitoring Data](https://mlsanity.com/paper/2401.15337/) (2024)
- [Deep Imbalanced Regression to Estimate Vascular Age from PPG Data: a Novel Digital Biomarker for Cardiovascular Health](https://mlsanity.com/paper/2406.14953/) (2024)
- [AnyECG-Lab: An Exploration Study of Fine-tuning an ECG Foundation Model to Estimate Laboratory Values from Single-Lead ECG Signals](https://mlsanity.com/paper/2510.22301/) (2025)
- [Fine-tuning an ECG Foundation Model to Predict Coronary CT Angiography Outcomes](https://mlsanity.com/paper/2512.05136/) (2025)
- [AnyECG: Evolved ECG Foundation Model for Holistic Health Profiling](https://mlsanity.com/paper/2601.10748/) (2026)

## Abstract (arXiv)

Deep learning models have achieved expert-level performance in healthcare with an exclusive focus on training accurate models. However, in many clinical environments such as intensive care unit (ICU), real-time model serving is equally if not more important than accuracy, because in ICU patient care is simultaneously more urgent and more expensive. Clinical decisions and their timeliness, therefore, directly affect both the patient outcome and the cost of care. To make timely decisions, we argue the underlying serving system must be latency-aware. To compound the challenge, health analytic applications often require a combination of models instead of a single model, to better specialize individual models for different targets, multi-modal data, different prediction windows, and potentially personalized predictions. To address these challenges, we propose HOLMES-an online model ensemble serving framework for healthcare applications. HOLMES dynamically identifies the best performing set of models to ensemble for highest accuracy, while also satisfying sub-second latency constraints on end-to-end prediction. We demonstrate that HOLMES is able to navigate the accuracy/latency tradeoff efficiently, compose the ensemble, and serve the model ensemble pipeline, scaling to simultaneously streaming data from 100 patients, each producing waveform data at 250~Hz. HOLMES outperforms the conventional offline batch-processed inference for the same clinical task in terms of accuracy and latency (by order of magnitude). HOLMES is tested on risk prediction task on pediatric cardio ICU data with above 95% prediction accuracy and sub-second latency on 64-bed simulation.