Event-Based Contrastive Learning for Medical Time Series
Not on the map. Why?
Abstract (arXiv)
In clinical practice, one often needs to identify whether a patient is at high risk of adverse outcomes after some key medical event. For example, quantifying the risk of adverse outcomes after an acute cardiovascular event helps healthcare providers identify those patients at the highest risk of poor outcomes; i.e., patients who benefit from invasive therapies that can lower their risk. Assessing the risk of adverse outcomes, however, is challenging due to the complexity, variability, and heterogeneity of longitudinal medical data, especially for individuals suffering from chronic diseases like heart failure. In this paper, we introduce Event-Based Contrastive Learning (EBCL) - a method for learning embeddings of heterogeneous patient data that preserves temporal information before and after key index events. We demonstrate that EBCL can be used to construct models that yield improved performance on important downstream tasks relative to other pretraining methods. We develop and test the method using a cohort of heart failure patients obtained from a large hospital network and the publicly available MIMIC-IV dataset consisting of patients in an intensive care unit at a large tertiary care center. On both cohorts, EBCL pretraining yields models that are performant with respect to a number of downstream tasks, including mortality, hospital readmission, and length of stay. In addition, unsupervised EBCL embeddings effectively cluster heart failure patients into subgroups with distinct outcomes, thereby providing information that helps identify new heart failure phenotypes. The contrastive framework around the index event can be adapted to a wide array of time-series datasets and provides information that can be used to guide personalized care.
Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Builds on 42 of its 66 references that are in the map. First posted on arXiv 16 Dec 2023.
Builds on
42 of 66 references.
References · 42
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
- 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,166
- 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,898
- 2020Contrastive Learning of Medical Visual Representations from Paired Images and Textcited by 641
- 2022Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learningcited by 561
- 2021Tabular Data: Deep Learning is Not All You Needcited by 535
- 2021Time-Series Representation Learning via Temporal and Contextual Contrastingcited by 345
- 2018Scalable and accurate deep learning for electronic health recordscited by 280
- 2019Unsupervised Scalable Representation Learning for Multivariate Time Seriescited by 269
30 more
- 2019BEHRT: Transformer for Electronic Health Recordscited by 263
- 2021Unsupervised Representation Learning for Time Series with Temporal Neighborhood Codingcited by 224
- 2015Doctor AI: Predicting Clinical Events via Recurrent Neural Networkscited by 208
- 2018A System for Massively Parallel Hyperparameter Tuningcited by 202
- 2020CLOCS: Contrastive Learning of Cardiac Signals Across Space, Time, and Patientscited by 171
- 2021TCLR: Temporal Contrastive Learning for Video Representationcited by 136
- 2019MIMIC-Extract: A Data Extraction, Preprocessing, and Representation Pipeline for MIMIC-IIIcited by 122
- 2021Graph-Guided Network for Irregularly Sampled Multivariate Time Seriescited by 111
- 2021Self-Supervised Transformer for Sparse and Irregularly Sampled Multivariate Clinical Time-Seriescited by 96
- 2020Language Models Are An Effective Patient Representation Learning Technique For Electronic Health Record Datacited by 87
- 2020Subject-Aware Contrastive Learning for Biosignalscited by 76
- 2023Identifiability Results for Multimodal Contrastive Learningcited by 61
- 2021Hi-BEHRT: Hierarchical Transformer-based model for accurate prediction of clinical events using multimodal longitudinal electronic health recordscited by 54
- 2023Event Stream GPT: A Data Pre-processing and Modeling Library for Generative, Pre-trained Transformers over Continuous-time Sequences of Complex Eventscited by 46
- 2020A Survey on Principles, Models and Methods for Learning from Irregularly Sampled Time Seriescited by 42
- 2020Learning from Irregularly-Sampled Time Series: A Missing Data Perspectivecited by 40
- 2022Lead-agnostic Self-supervised Learning for Local and Global Representations of Electrocardiogramcited by 38
- 20213KG: Contrastive Learning of 12-Lead Electrocardiograms using Physiologically-Inspired Augmentationscited by 36
- 2021Patient Contrastive Learning: a Performant, Expressive, and Practical Approach to ECG Modelingcited by 25
- 2023Learning Missing Modal Electronic Health Records with Unified Multi-modal Data Embedding and Modality-Aware Attentioncited by 25
- 2015Feed-Forward Networks with Attention Can Solve Some Long-Term Memory Problemscited by 23
- 2023Multimodal Pretraining of Medical Time Series and Notescited by 22
- 2023DuETT: Dual Event Time Transformer for Electronic Health Recordscited by 17
- 2023Sequential Multi-Dimensional Self-Supervised Learning for Clinical Time Seriescited by 16
- 2021Structure Inducing Pre-Trainingcited by 10
- 2020Clinical Risk Prediction with Temporal Probabilistic Asymmetric Multi-Task Learningcited by 8
- 2023Clinical Decision Transformer: Intended Treatment Recommendation through Goal Promptingcited by 8
- 2021Predicting Intensive Care Unit Length of Stay and Mortality Using Patient Vital Signs: Machine Learning Model Development and Validationcited by 7
- 2021Leveraging Time Irreversibility with Order-Contrastive Pre-trainingcited by 7
- 2023Deep Metric Learning for the Hemodynamics Inference with Electrocardiogram Signalscited by 4
Built on by
Citing papers per year: 2024: 3; 2025: 7; 2026: 3.
13 papers.
Sort:
- 2025Pulse-PPG: An Open-Source Field-Trained PPG Foundation Model for Wearable Applications Across Lab and Field Settingscited by 37
- 2024RelCon: Relative Contrastive Learning for a Motion Foundation Model for Wearable Datacited by 20
- 2025Foundation Model of Electronic Medical Records for Adaptive Risk Estimationcited by 19
- 2025Beyond Sensor Data: Foundation Models of Behavioral Data from Wearables Improve Health Predictionscited by 18
- 2025Memorize and Rank: Elevating Large Language Models for Clinical Diagnosis Predictioncited by 8
- 2025FoMoH: A clinically meaningful foundation model evaluation for structured electronic health recordscited by 6
- 2024MEDS-Tab: Automated tabularization and baseline methods for MEDS datasetscited by 4
- 2024Recent Advances, Applications, and Open Challenges in Machine Learning for Health: Reflections from Research Roundtables at ML4H 2023 Symposiumcited by 3
- 2025Time-Aware Attention for Enhanced Electronic Health Records Modelingcited by 3
- 2026The Taxonomies, Training, and Applications of Event Stream Modelling for Electronic Health Recordscited by 3
- 2025Towards Unified Approaches in Self-Supervised Event Stream Modeling: Progress and Prospectscited by 0
- 2026NEST: Nested Event Stream Transformer for Sequences of Multisetscited by 0
1 more
- 2026The Taxonomies, Training, and Applications of Event Stream Modelling for Electronic Health Records14 Mar 2026
- 2026Learning Representations from Incomplete EHR Data with Dual-Masked Autoencoding16 Feb 2026
- 2026NEST: Nested Event Stream Transformer for Sequences of Multisets31 Jan 2026
- 2025Time-Aware Attention for Enhanced Electronic Health Records Modeling20 Jul 2025
- 2025Beyond Sensor Data: Foundation Models of Behavioral Data from Wearables Improve Health Predictions30 Jun 2025
- 2025FoMoH: A clinically meaningful foundation model evaluation for structured electronic health records22 May 2025
- 2025Foundation Model of Electronic Medical Records for Adaptive Risk Estimation10 Feb 2025
- 2025Towards Unified Approaches in Self-Supervised Event Stream Modeling: Progress and Prospects7 Feb 2025
- 2025Pulse-PPG: An Open-Source Field-Trained PPG Foundation Model for Wearable Applications Across Lab and Field Settings3 Feb 2025
- 2025Memorize and Rank: Elevating Large Language Models for Clinical Diagnosis Prediction28 Jan 2025
- 2024RelCon: Relative Contrastive Learning for a Motion Foundation Model for Wearable Data27 Nov 2024
- 2024MEDS-Tab: Automated tabularization and baseline methods for MEDS datasets31 Oct 2024
- 2025Pulse-PPG: An Open-Source Field-Trained PPG Foundation Model for Wearable Applications Across Lab and Field Settingscited by 37
- 2024RelCon: Relative Contrastive Learning for a Motion Foundation Model for Wearable Datacited by 20
- 2025Foundation Model of Electronic Medical Records for Adaptive Risk Estimationcited by 19
- 2025Beyond Sensor Data: Foundation Models of Behavioral Data from Wearables Improve Health Predictionscited by 18
- 2025Memorize and Rank: Elevating Large Language Models for Clinical Diagnosis Predictioncited by 8
- 2025FoMoH: A clinically meaningful foundation model evaluation for structured electronic health recordscited by 6
- 2024MEDS-Tab: Automated tabularization and baseline methods for MEDS datasetscited by 4
- 2024Recent Advances, Applications, and Open Challenges in Machine Learning for Health: Reflections from Research Roundtables at ML4H 2023 Symposiumcited by 3
- 2025Time-Aware Attention for Enhanced Electronic Health Records Modelingcited by 3
- 2026The Taxonomies, Training, and Applications of Event Stream Modelling for Electronic Health Recordscited by 3
- 2025Towards Unified Approaches in Self-Supervised Event Stream Modeling: Progress and Prospectscited by 0
- 2026NEST: Nested Event Stream Transformer for Sequences of Multisetscited by 0
12 most cited of 13 All 13 (JSON)
- 2026The Taxonomies, Training, and Applications of Event Stream Modelling for Electronic Health Records14 Mar 2026
- 2026Learning Representations from Incomplete EHR Data with Dual-Masked Autoencoding16 Feb 2026
- 2026NEST: Nested Event Stream Transformer for Sequences of Multisets31 Jan 2026
- 2025Time-Aware Attention for Enhanced Electronic Health Records Modeling20 Jul 2025
- 2025Beyond Sensor Data: Foundation Models of Behavioral Data from Wearables Improve Health Predictions30 Jun 2025
- 2025FoMoH: A clinically meaningful foundation model evaluation for structured electronic health records22 May 2025
- 2025Foundation Model of Electronic Medical Records for Adaptive Risk Estimation10 Feb 2025
- 2025Towards Unified Approaches in Self-Supervised Event Stream Modeling: Progress and Prospects7 Feb 2025
- 2025Pulse-PPG: An Open-Source Field-Trained PPG Foundation Model for Wearable Applications Across Lab and Field Settings3 Feb 2025
- 2025Memorize and Rank: Elevating Large Language Models for Clinical Diagnosis Prediction28 Jan 2025
- 2024RelCon: Relative Contrastive Learning for a Motion Foundation Model for Wearable Data27 Nov 2024
- 2024MEDS-Tab: Automated tabularization and baseline methods for MEDS datasets31 Oct 2024
12 newest of 13 All 13 (JSON)