# A Laplace diffusion-based transformer model for heart rate forecasting within daily activity context

arXiv: https://arxiv.org/abs/2508.16655 · HTML: https://mlsanity.com/paper/2508.16655/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 27 of its 62 references that are in the map.
- First posted on arXiv 20 Aug 2025.

## Builds on (most cited first, 27 of 27)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [A Time Series is Worth 64 Words: Long-term Forecasting with Transformers](https://mlsanity.com/paper/2211.14730/) (2022)
- [Diffusion Models: A Comprehensive Survey of Methods and Applications](https://mlsanity.com/paper/2209.00796/) (2022)
- [CSDI: Conditional Score-based Diffusion Models for Probabilistic Time Series Imputation](https://mlsanity.com/paper/2107.03502/) (2021)
- [ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmission](https://mlsanity.com/paper/1904.05342/) (2019)
- [Med-BERT: pre-trained contextualized embeddings on large-scale structured electronic health records for disease prediction](https://mlsanity.com/paper/2005.12833/) (2020)
- [BEHRT: Transformer for Electronic Health Records](https://mlsanity.com/paper/1907.09538/) (2019)
- [Diffusion-based Time Series Imputation and Forecasting with Structured State Space Models](https://mlsanity.com/paper/2208.09399/) (2022)
- [Large language models in healthcare and medical domain: A review](https://mlsanity.com/paper/2401.06775/) (2023)
- [Deep Isolation Forest for Anomaly Detection](https://mlsanity.com/paper/2206.06602/) (2022)
- [Shape and Time Distortion Loss for Training Deep Time Series Forecasting Models](https://mlsanity.com/paper/1909.09020/) (2019)
- [A Survey on Diffusion Models for Time Series and Spatio-Temporal Data](https://mlsanity.com/paper/2404.18886/) (2024)
- [Time Series Modeling for Heart Rate Prediction: From ARIMA to Transformers](https://mlsanity.com/paper/2406.12199/) (2024)
- [A Comprehensive Overview and Comparative Analysis on Deep Learning Models: CNN, RNN, LSTM, GRU](https://mlsanity.com/paper/2305.17473/) (2023)
- [Transformers in Healthcare: A Survey](https://mlsanity.com/paper/2307.00067/) (2023)
- [A Transformer-based Diffusion Probabilistic Model for Heart Rate and Blood Pressure Forecasting in Intensive Care Unit](https://mlsanity.com/paper/2301.06625/) (2023)
- [A Survey on Diffusion Models for Anomaly Detection](https://mlsanity.com/paper/2501.11430/) (2025)
- [An explainable Transformer-based deep learning model for the prediction of incident heart failure](https://mlsanity.com/paper/2101.11359/) (2021)
- [Assessing the capacity of a denoising diffusion probabilistic model to reproduce spatial context](https://mlsanity.com/paper/2309.10817/) (2023)
- [Improving Diffusion Models for ECG Imputation with an Augmented Template Prior](https://mlsanity.com/paper/2310.15742/) (2023)
- [Transformer-Based Behavioral Representation Learning Enables Transfer Learning for Mobile Sensing in Small Datasets](https://mlsanity.com/paper/2107.06097/) (2021)
- [Multi-modal Heart Failure Risk Estimation based on Short ECG and Sampled Long-Term HRV](https://mlsanity.com/paper/2403.15408/) (2024)
- [New care pathways for supporting transitional care from hospitals to home using AI and personalized digital assistance](https://mlsanity.com/paper/2504.13877/) (2025)

## Abstract (arXiv)

With the advent of wearable Internet of Things (IoT) devices, remote patient monitoring (RPM) emerged as a promising solution for managing heart failure. However, the heart rate can fluctuate significantly due to various factors, and without correlating it to the patient's actual physical activity, it becomes difficult to assess whether changes are significant. Although Artificial Intelligence (AI) models may enhance the accuracy and contextual understanding of remote heart rate monitoring, the integration of activity data is still rarely addressed. In this paper, we propose a Transformer model combined with a Laplace diffusion technique to model heart rate fluctuations driven by physical activity of the patient. Unlike prior models that treat activity as secondary, our approach conditions the entire modeling process on activity context using specialized embeddings and attention mechanisms to prioritize activity specific historical patents. The model captures both long-term patterns and activity-specific heart rate dynamics by incorporating contextualized embeddings and dedicated encoder. The Transformer model was validated on a real-world dataset collected from 29 patients over a 4-month period. Experimental results show that our model outperforms current state-of-the-art methods, achieving a 43% reduction in mean absolute error compared to the considered baseline models. Moreover, the coefficient of determination R2 is 0.97 indicating the model predicted heart rate is in strong agreement with actual heart rate values. These findings suggest that the proposed model is a practical and effective tool for supporting both healthcare providers and remote patient monitoring systems.