# Patient Contrastive Learning: a Performant, Expressive, and Practical Approach to ECG Modeling

arXiv: https://arxiv.org/abs/2104.04569 · HTML: https://mlsanity.com/paper/2104.04569/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 41 references that are in the map.
- First posted on arXiv 9 Apr 2021.

## Builds on (most cited first, 15 of 15)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [What Makes for Good Views for Contrastive Learning?](https://mlsanity.com/paper/2005.10243/) (2020)
- [Contrastive Learning of Medical Visual Representations from Paired Images and Text](https://mlsanity.com/paper/2010.00747/) (2020)
- [Network In Network](https://mlsanity.com/paper/1312.4400/) (2013)
- [Big Self-Supervised Models Advance Medical Image Classification](https://mlsanity.com/paper/2101.05224/) (2021)
- [Contrastive learning of global and local features for medical image segmentation with limited annotations](https://mlsanity.com/paper/2006.10511/) (2020)
- [Revisiting Self-Supervised Visual Representation Learning](https://mlsanity.com/paper/1901.09005/) (2019)
- [CLOCS: Contrastive Learning of Cardiac Signals Across Space, Time, and Patients](https://mlsanity.com/paper/2005.13249/) (2020)
- [Automatic diagnosis of the 12-lead ECG using a deep neural network](https://mlsanity.com/paper/1904.01949/) (2019)
- [Subject-Aware Contrastive Learning for Biosignals](https://mlsanity.com/paper/2007.04871/) (2020)
- [Cardiologist-Level Arrhythmia Detection with Convolutional Neural Networks](https://mlsanity.com/paper/1707.01836/) (2017)
- [Self-Supervised Learning for Spinal MRIs](https://mlsanity.com/paper/1708.00367/) (2017)

## Built on by (most cited first, 25 of 25)

- [Large-scale Training of Foundation Models for Wearable Biosignals](https://mlsanity.com/paper/2312.05409/) (2023)
- [RelCon: Relative Contrastive Learning for a Motion Foundation Model for Wearable Data](https://mlsanity.com/paper/2411.18822/) (2024)
- [Event-Based Contrastive Learning for Medical Time Series](https://mlsanity.com/paper/2312.10308/) (2023)
- [Clinically Labeled Contrastive Learning for OCT Biomarker Classification](https://mlsanity.com/paper/2305.15154/) (2023)
- [Leveraging Time Irreversibility with Order-Contrastive Pre-training](https://mlsanity.com/paper/2111.02599/) (2021)
- [Deep Metric Learning for the Hemodynamics Inference with Electrocardiogram Signals](https://mlsanity.com/paper/2308.04650/) (2023)
- [Contrastive Learning Is Not Optimal for Quasiperiodic Time Series](https://mlsanity.com/paper/2407.17073/) (2024)
- [Beyond Patient Invariance: Learning Cardiac Dynamics via Action-Conditioned JEPAs](https://mlsanity.com/paper/2604.22618/) (2026)
- [Unlocking Telemetry Potential: Self-Supervised Learning for Continuous Clinical Electrocardiogram Monitoring](https://mlsanity.com/paper/2406.16915/) (2024)
- [Estimating ECG Intervals from Lead-I Alone: External Validation of Supervised Models](https://mlsanity.com/paper/2408.15272/) (2024)
- [Enhancing Contrastive Learning-based Electrocardiogram Pretrained Model with Patient Memory Queue](https://mlsanity.com/paper/2506.06310/) (2025)
- [How Do Electrocardiogram Models Scale?](https://mlsanity.com/paper/2605.17276/) (2026)
- [Self-Supervised Mental Disorder Classifiers via Time Reversal](https://mlsanity.com/paper/2211.16398/) (2022)
- [Subject-based Non-contrastive Self-Supervised Learning for ECG Signal Processing](https://mlsanity.com/paper/2305.10347/) (2023)
- [A Comparison of Self-Supervised Pretraining Approaches for Predicting Disease Risk from Chest Radiograph Images](https://mlsanity.com/paper/2306.08955/) (2023)
- [Learning Beyond Similarities: Incorporating Dissimilarities between Positive Pairs in Self-Supervised Time Series Learning](https://mlsanity.com/paper/2309.07526/) (2023)
- [CuPID: Leveraging Masked Single-Lead ECG Modelling for Enhancing the Representations](https://mlsanity.com/paper/2502.21127/) (2025)
- [Parallel-Learning of Invariant and Tempo-variant Attributes of Single-Lead Cardiac Signals: PLITA](https://mlsanity.com/paper/2502.21162/) (2025)
- [Leveraging Self-Supervised Learning Methods for Remote Screening of Subjects with Paroxysmal Atrial Fibrillation](https://mlsanity.com/paper/2503.02621/) (2025)
- [Position: Evaluation of ECG Representations Must Be Fixed](https://mlsanity.com/paper/2602.17531/) (2026)
- [Patient-Aware Contrastive Learning Preserves Per-Patient Structure in RR-Interval Representations](https://mlsanity.com/paper/2606.23570/) (2026)
- [The Impact of Temporal Context Length and Encoding Strategies on Self-Supervised ECG Representation Learning](https://mlsanity.com/paper/2608.12695/) (2026)
- [Beat-Synchronous Tokenization for ECG Transformers](https://mlsanity.com/paper/2608.30367/) (2026)
- [Learning Cardiac Features: ECG Biometrics Across Time and~Exercise](https://mlsanity.com/paper/2609.21962/) (2026)
- [Training and Scaling Compute-Optimal Physiological Waveform Foundation Models](https://mlsanity.com/paper/2610.05649/) (2026)

## Abstract (arXiv)

Supervised machine learning applications in health care are often limited due to a scarcity of labeled training data. To mitigate this effect of small sample size, we introduce a pre-training approach, Patient Contrastive Learning of Representations (PCLR), which creates latent representations of ECGs from a large number of unlabeled examples. The resulting representations are expressive, performant, and practical across a wide spectrum of clinical tasks. We develop PCLR using a large health care system with over 3.2 million 12-lead ECGs, and demonstrate substantial improvements across multiple new tasks when there are fewer than 5,000 labels. We release our model to extract ECG representations at https://github.com/broadinstitute/ml4h/tree/master/model_zoo/PCLR.