# 3KG: Contrastive Learning of 12-Lead Electrocardiograms using Physiologically-Inspired Augmentations

arXiv: https://arxiv.org/abs/2106.04452 · HTML: https://mlsanity.com/paper/2106.04452/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 44 references that are in the map.
- First posted on arXiv 21 Apr 2021.

## Builds on (most cited first, 21 of 21)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [Big Self-Supervised Models are Strong Semi-Supervised Learners](https://mlsanity.com/paper/2006.10029/) (2020)
- [Contrastive Multiview Coding](https://mlsanity.com/paper/1906.05849/) (2019)
- [Learning deep representations by mutual information estimation and maximization](https://mlsanity.com/paper/1808.06670/) (2018)
- [What Makes for Good Views for Contrastive Learning?](https://mlsanity.com/paper/2005.10243/) (2020)
- [Cyclical Learning Rates for Training Neural Networks](https://mlsanity.com/paper/1506.01186/) (2015)
- [CLOCS: Contrastive Learning of Cardiac Signals Across Space, Time, and Patients](https://mlsanity.com/paper/2005.13249/) (2020)
- [Self-supervised ECG Representation Learning for Emotion Recognition](https://mlsanity.com/paper/2002.03898/) (2020)
- [Self-supervised representation learning from 12-lead ECG data](https://mlsanity.com/paper/2103.12676/) (2021)
- [Subject-Aware Contrastive Learning for Biosignals](https://mlsanity.com/paper/2007.04871/) (2020)
- [MoCo-CXR: MoCo Pretraining Improves Representation and Transferability of Chest X-ray Models](https://mlsanity.com/paper/2010.05352/) (2020)
- [MedAug: Contrastive learning leveraging patient metadata improves representations for chest X-ray interpretation](https://mlsanity.com/paper/2102.10663/) (2021)
- [Self-supervised Learning for ECG-based Emotion Recognition](https://mlsanity.com/paper/1910.07497/) (2019)
- [Automated and Interpretable Patient ECG Profiles for Disease Detection, Tracking, and Discovery](https://mlsanity.com/paper/1807.02569/) (2018)
- [PCPs: Patient Cardiac Prototypes](https://mlsanity.com/paper/2011.14227/) (2020)

## Built on by (most cited first, 30 of 36)

- [Large-scale Training of Foundation Models for Wearable Biosignals](https://mlsanity.com/paper/2312.05409/) (2023)
- [Guiding Masked Representation Learning to Capture Spatio-Temporal Relationship of Electrocardiogram](https://mlsanity.com/paper/2402.09450/) (2024)
- [ECG-QA: A Comprehensive Question Answering Dataset Combined With Electrocardiogram](https://mlsanity.com/paper/2306.15681/) (2023)
- [SleepFM: Multi-modal Representation Learning for Sleep Across Brain Activity, ECG and Respiratory Signals](https://mlsanity.com/paper/2405.17766/) (2024)
- [Lead-agnostic Self-supervised Learning for Local and Global Representations of Electrocardiogram](https://mlsanity.com/paper/2203.06889/) (2022)
- [ECG Semantic Integrator (ESI): A Foundation ECG Model Pretrained with LLM-Enhanced Cardiological Text](https://mlsanity.com/paper/2405.19366/) (2024)
- [RelCon: Relative Contrastive Learning for a Motion Foundation Model for Wearable Data](https://mlsanity.com/paper/2411.18822/) (2024)
- [Boosting Masked ECG-Text Auto-Encoders as Discriminative Learners](https://mlsanity.com/paper/2410.02131/) (2024)
- [Sequential Multi-Dimensional Self-Supervised Learning for Clinical Time Series](https://mlsanity.com/paper/2307.10923/) (2023)
- [Wearable Accelerometer Foundation Models for Health via Knowledge Distillation](https://mlsanity.com/paper/2412.11276/) (2024)
- [Towards quantitative precision for ECG analysis: Leveraging state space models, self-supervision and patient metadata](https://mlsanity.com/paper/2308.15291/) (2023)
- [Event-Based Contrastive Learning for Medical Time Series](https://mlsanity.com/paper/2312.10308/) (2023)
- [Data Augmentation for Electrocardiograms](https://mlsanity.com/paper/2204.04360/) (2022)
- [Analysis of Augmentations for Contrastive ECG Representation Learning](https://mlsanity.com/paper/2206.07656/) (2022)
- [ECG-Byte: A Tokenizer for End-to-End Generative Electrocardiogram Language Modeling](https://mlsanity.com/paper/2412.14373/) (2024)
- [In-Distribution and Out-of-Distribution Self-supervised ECG Representation Learning for Arrhythmia Detection](https://mlsanity.com/paper/2304.06427/) (2023)
- [Electrocardiogram-Language Model for Few-Shot Question Answering with Meta Learning](https://mlsanity.com/paper/2410.14464/) (2024)
- [Deep Metric Learning for the Hemodynamics Inference with Electrocardiogram Signals](https://mlsanity.com/paper/2308.04650/) (2023)
- [Self-Alignment Learning to Improve Myocardial Infarction Detection from Single-Lead ECG](https://mlsanity.com/paper/2509.19397/) (2025)
- [Pretraining ECG Data with Adversarial Masking Improves Model Generalizability for Data-Scarce Tasks](https://mlsanity.com/paper/2211.07889/) (2022)
- [Self-DANA: A Resource-Efficient Channel-Adaptive Self-Supervised Approach for ECG Foundation Models](https://mlsanity.com/paper/2507.14151/) (2025)
- [LEAVES: Learning Views for Time-Series Biobehavioral Data in Contrastive Learning](https://mlsanity.com/paper/2210.07340/) (2022)
- [Which Augmentation Should I Use? An Empirical Investigation of Augmentations for Self-Supervised Phonocardiogram Representation Learning](https://mlsanity.com/paper/2312.00502/) (2023)
- [Riemannian Prediction of Anatomical Diagnoses in Congenital Heart Disease based on 12-lead ECGs](https://mlsanity.com/paper/2312.09437/) (2023)
- [OC-Distill: Ontology-aware Contrastive Learning with Cross-Modal Distillation for ICU Risk Prediction](https://mlsanity.com/paper/2604.16878/) (2026)
- [Automated Cardiovascular Record Retrieval by Multimodal Learning between Electrocardiogram and Clinical Report](https://mlsanity.com/paper/2304.06286/) (2023)
- [Finding "Good Views" of Electrocardiogram Signals for Inferring Abnormalities in Cardiac Condition](https://mlsanity.com/paper/2411.17702/) (2024)
- [Learning ECG Representations via Poly-Window Contrastive Learning](https://mlsanity.com/paper/2508.15225/) (2025)
- [ECG-Soup: Harnessing Multi-Layer Synergy for ECG Foundation Models](https://mlsanity.com/paper/2509.00102/) (2025)
- [Domain Knowledge is Power: Leveraging Physiological Priors for Self Supervised Representation Learning in Electrocardiography](https://mlsanity.com/paper/2509.08116/) (2025)

## Abstract (arXiv)

We propose 3KG, a physiologically-inspired contrastive learning approach that generates views using 3D augmentations of the 12-lead electrocardiogram. We evaluate representation quality by fine-tuning a linear layer for the downstream task of 23-class diagnosis on the PhysioNet 2020 challenge training data and find that 3KG achieves a 9.1% increase in mean AUC over the best self-supervised baseline when trained on 1% of labeled data. Our empirical analysis shows that combining spatial and temporal augmentations produces the strongest representations. In addition, we investigate the effect of this physiologically-inspired pretraining on downstream performance on different disease subgroups and find that 3KG makes the greatest gains for conduction and rhythm abnormalities. Our method allows for flexibility in incorporating other self-supervised strategies and highlights the potential for similar modality-specific augmentations for other biomedical signals.