# A Transformer-based Diffusion Probabilistic Model for Heart Rate and Blood Pressure Forecasting in Intensive Care Unit

arXiv: https://arxiv.org/abs/2301.06625 · HTML: https://mlsanity.com/paper/2301.06625/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 44 references that are in the map.
- First posted on arXiv 16 Jan 2023.

## Builds on (most cited first, 25 of 25)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016)
- [Structured Denoising Diffusion Models in Discrete State-Spaces](https://mlsanity.com/paper/2107.03006/) (2021)
- [DiffWave: A Versatile Diffusion Model for Audio Synthesis](https://mlsanity.com/paper/2009.09761/) (2020)
- [Diffusion Models: A Comprehensive Survey of Methods and Applications](https://mlsanity.com/paper/2209.00796/) (2022)
- [DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks](https://mlsanity.com/paper/1704.04110/) (2017)
- [CSDI: Conditional Score-based Diffusion Models for Probabilistic Time Series Imputation](https://mlsanity.com/paper/2107.03502/) (2021)
- [Understanding Diffusion Models: A Unified Perspective](https://mlsanity.com/paper/2208.11970/) (2022)
- [Autoregressive Denoising Diffusion Models for Multivariate Probabilistic Time Series Forecasting](https://mlsanity.com/paper/2101.12072/) (2021)
- [Advances in Variational Inference](https://mlsanity.com/paper/1711.05597/) (2017)
- [A Multi-Horizon Quantile Recurrent Forecaster](https://mlsanity.com/paper/1711.11053/) (2017)
- [Protein Structure and Sequence Generation with Equivariant Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2205.15019/) (2022)
- [Self-Supervised Transformer for Sparse and Irregularly Sampled Multivariate Clinical Time-Series](https://mlsanity.com/paper/2107.14293/) (2021)
- [Deep Factors for Forecasting](https://mlsanity.com/paper/1905.12417/) (2019)
- [Ensemble Conformalized Quantile Regression for Probabilistic Time Series Forecasting](https://mlsanity.com/paper/2202.08756/) (2022)
- [Threat Model-Agnostic Adversarial Defense using Diffusion Models](https://mlsanity.com/paper/2207.08089/) (2022)
- [Recurrent neural networks with specialized word embeddings for health-domain named-entity recognition](https://mlsanity.com/paper/1706.09569/) (2017)

## Built on by (most cited first, 14 of 14)

- [DiffSTG: Probabilistic Spatio-Temporal Graph Forecasting with Denoising Diffusion Models](https://mlsanity.com/paper/2301.13629/) (2023)
- [A Survey on Diffusion Models for Time Series and Spatio-Temporal Data](https://mlsanity.com/paper/2404.18886/) (2024)
- [Time Series Modeling for Heart Rate Prediction: From ARIMA to Transformers](https://mlsanity.com/paper/2406.12199/) (2024)
- [The Rise of Diffusion Models in Time-Series Forecasting](https://mlsanity.com/paper/2401.03006/) (2024)
- [A Comprehensive Survey on Generative Diffusion Models for Structured Data](https://mlsanity.com/paper/2306.04139/) (2023)
- [Versatile Cardiovascular Signal Generation with a Unified Diffusion Transformer](https://mlsanity.com/paper/2505.22306/) (2025)
- [TimeWak: Temporal Chained-Hashing Watermark for Time Series Data](https://mlsanity.com/paper/2506.06407/) (2025)
- [MultiWave: Multiresolution Deep Architectures through Wavelet Decomposition for Multivariate Time Series Prediction](https://mlsanity.com/paper/2306.10164/) (2023)
- [Enhancing Exchange Rate Forecasting with Explainable Deep Learning Models](https://mlsanity.com/paper/2410.19241/) (2024)
- [Towards Trustworthy Vital Sign Forecasting: Leveraging Uncertainty for Prediction Intervals](https://mlsanity.com/paper/2509.01319/) (2025)
- [EAMDrift: An interpretable self retrain model for time series](https://mlsanity.com/paper/2305.19837/) (2023)
- [Deep Inverse Design for High-Level Synthesis](https://mlsanity.com/paper/2407.08797/) (2024)
- [A Laplace diffusion-based transformer model for heart rate forecasting within daily activity context](https://mlsanity.com/paper/2508.16655/) (2025)
- [Pattern-Guided Diffusion Models](https://mlsanity.com/paper/2512.13945/) (2025)

## Abstract (arXiv)

Background and Objective: Vital sign monitoring in the Intensive Care Unit (ICU) is crucial for enabling prompt interventions for patients. This underscores the need for an accurate predictive system. Therefore, this study proposes a novel deep learning approach for forecasting Heart Rate (HR), Systolic Blood Pressure (SBP), and Diastolic Blood Pressure (DBP) in the ICU. Methods: We extracted 24,886 ICU stays from the MIMIC-III database which contains data from over 46 thousand patients, to train and test the model. The model proposed in this study, Transformer-based Diffusion Probabilistic Model for Sparse Time Series Forecasting (TDSTF), merges Transformer and diffusion models to forecast vital signs. The TDSTF model showed state-of-the-art performance in predicting vital signs in the ICU, outperforming other models' ability to predict distributions of vital signs and being more computationally efficient. The code is available at https://github.com/PingChang818/TDSTF. Results: The results of the study showed that TDSTF achieved a Standardized Average Continuous Ranked Probability Score (SACRPS) of 0.4438 and a Mean Squared Error (MSE) of 0.4168, an improvement of 18.9% and 34.3% over the best baseline model, respectively. The inference speed of TDSTF is more than 17 times faster than the best baseline model. Conclusion: TDSTF is an effective and efficient solution for forecasting vital signs in the ICU, and it shows a significant improvement compared to other models in the field.