mlsanityData to 14 Oct 2026

Event-Based Contrastive Learning for Medical Time Series

arXiv 2312.10308 ↗First posted 16 Dec 2023cs.LGcited by 13

Not on the map. Why?

Abstract (arXiv)

In clinical practice, one often needs to identify whether a patient is at high risk of adverse outcomes after some key medical event. For example, quantifying the risk of adverse outcomes after an acute cardiovascular event helps healthcare providers identify those patients at the highest risk of poor outcomes; i.e., patients who benefit from invasive therapies that can lower their risk. Assessing the risk of adverse outcomes, however, is challenging due to the complexity, variability, and heterogeneity of longitudinal medical data, especially for individuals suffering from chronic diseases like heart failure. In this paper, we introduce Event-Based Contrastive Learning (EBCL) - a method for learning embeddings of heterogeneous patient data that preserves temporal information before and after key index events. We demonstrate that EBCL can be used to construct models that yield improved performance on important downstream tasks relative to other pretraining methods. We develop and test the method using a cohort of heart failure patients obtained from a large hospital network and the publicly available MIMIC-IV dataset consisting of patients in an intensive care unit at a large tertiary care center. On both cohorts, EBCL pretraining yields models that are performant with respect to a number of downstream tasks, including mortality, hospital readmission, and length of stay. In addition, unsupervised EBCL embeddings effectively cluster heart failure patients into subgroups with distinct outcomes, thereby providing information that helps identify new heart failure phenotypes. The contrastive framework around the index event can be adapted to a wide array of time-series datasets and provides information that can be used to guide personalized care.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Builds on 42 of its 66 references that are in the map. First posted on arXiv 16 Dec 2023.

Builds on

42 of 66 references.

References · 42

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  5. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,166
  6. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,898
  7. 2020Contrastive Learning of Medical Visual Representations from Paired Images and Textcited by 641
  8. 2022Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learningcited by 561
  9. 2021Tabular Data: Deep Learning is Not All You Needcited by 535
  10. 2021Time-Series Representation Learning via Temporal and Contextual Contrastingcited by 345
  11. 2018Scalable and accurate deep learning for electronic health recordscited by 280
  12. 2019Unsupervised Scalable Representation Learning for Multivariate Time Seriescited by 269
30 more
  1. 2019BEHRT: Transformer for Electronic Health Recordscited by 263
  2. 2021Unsupervised Representation Learning for Time Series with Temporal Neighborhood Codingcited by 224
  3. 2015Doctor AI: Predicting Clinical Events via Recurrent Neural Networkscited by 208
  4. 2018A System for Massively Parallel Hyperparameter Tuningcited by 202
  5. 2020CLOCS: Contrastive Learning of Cardiac Signals Across Space, Time, and Patientscited by 171
  6. 2021TCLR: Temporal Contrastive Learning for Video Representationcited by 136
  7. 2019MIMIC-Extract: A Data Extraction, Preprocessing, and Representation Pipeline for MIMIC-IIIcited by 122
  8. 2021Graph-Guided Network for Irregularly Sampled Multivariate Time Seriescited by 111
  9. 2021Self-Supervised Transformer for Sparse and Irregularly Sampled Multivariate Clinical Time-Seriescited by 96
  10. 2020Language Models Are An Effective Patient Representation Learning Technique For Electronic Health Record Datacited by 87
  11. 2020Subject-Aware Contrastive Learning for Biosignalscited by 76
  12. 2023Identifiability Results for Multimodal Contrastive Learningcited by 61
  13. 2021Hi-BEHRT: Hierarchical Transformer-based model for accurate prediction of clinical events using multimodal longitudinal electronic health recordscited by 54
  14. 2023Event Stream GPT: A Data Pre-processing and Modeling Library for Generative, Pre-trained Transformers over Continuous-time Sequences of Complex Eventscited by 46
  15. 2020A Survey on Principles, Models and Methods for Learning from Irregularly Sampled Time Seriescited by 42
  16. 2020Learning from Irregularly-Sampled Time Series: A Missing Data Perspectivecited by 40
  17. 2022Lead-agnostic Self-supervised Learning for Local and Global Representations of Electrocardiogramcited by 38
  18. 20213KG: Contrastive Learning of 12-Lead Electrocardiograms using Physiologically-Inspired Augmentationscited by 36
  19. 2021Patient Contrastive Learning: a Performant, Expressive, and Practical Approach to ECG Modelingcited by 25
  20. 2023Learning Missing Modal Electronic Health Records with Unified Multi-modal Data Embedding and Modality-Aware Attentioncited by 25
  21. 2015Feed-Forward Networks with Attention Can Solve Some Long-Term Memory Problemscited by 23
  22. 2023Multimodal Pretraining of Medical Time Series and Notescited by 22
  23. 2023DuETT: Dual Event Time Transformer for Electronic Health Recordscited by 17
  24. 2023Sequential Multi-Dimensional Self-Supervised Learning for Clinical Time Seriescited by 16
  25. 2021Structure Inducing Pre-Trainingcited by 10
  26. 2020Clinical Risk Prediction with Temporal Probabilistic Asymmetric Multi-Task Learningcited by 8
  27. 2023Clinical Decision Transformer: Intended Treatment Recommendation through Goal Promptingcited by 8
  28. 2021Predicting Intensive Care Unit Length of Stay and Mortality Using Patient Vital Signs: Machine Learning Model Development and Validationcited by 7
  29. 2021Leveraging Time Irreversibility with Order-Contrastive Pre-trainingcited by 7
  30. 2023Deep Metric Learning for the Hemodynamics Inference with Electrocardiogram Signalscited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 7; 2026: 3.

13 papers.

Sort: