# Lead-agnostic Self-supervised Learning for Local and Global Representations of Electrocardiogram

arXiv: https://arxiv.org/abs/2203.06889 · HTML: https://mlsanity.com/paper/2203.06889/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 26 references that are in the map.
- First posted on arXiv 14 Mar 2022.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019)
- [CLOCS: Contrastive Learning of Cardiac Signals Across Space, Time, and Patients](https://mlsanity.com/paper/2005.13249/) (2020)
- [Self-supervised representation learning from 12-lead ECG data](https://mlsanity.com/paper/2103.12676/) (2021)
- [ECG Heartbeat Classification: A Deep Transferable Representation](https://mlsanity.com/paper/1805.00794/) (2018)
- [3KG: Contrastive Learning of 12-Lead Electrocardiograms using Physiologically-Inspired Augmentations](https://mlsanity.com/paper/2106.04452/) (2021)

## Built on by (most cited first, 30 of 38)

- [ECG-FM: An Open Electrocardiogram Foundation Model](https://mlsanity.com/paper/2408.05178/) (2024)
- [Guiding Masked Representation Learning to Capture Spatio-Temporal Relationship of Electrocardiogram](https://mlsanity.com/paper/2402.09450/) (2024)
- [ECG-QA: A Comprehensive Question Answering Dataset Combined With Electrocardiogram](https://mlsanity.com/paper/2306.15681/) (2023)
- [Reading Your Heart: Learning ECG Words and Sentences via Pre-training ECG Language Model](https://mlsanity.com/paper/2502.10707/) (2025)
- [ECG Semantic Integrator (ESI): A Foundation ECG Model Pretrained with LLM-Enhanced Cardiological Text](https://mlsanity.com/paper/2405.19366/) (2024)
- [Learning General Representation of 12-Lead Electrocardiogram with a Joint-Embedding Predictive Architecture](https://mlsanity.com/paper/2410.08559/) (2024)
- [Boosting Masked ECG-Text Auto-Encoders as Discriminative Learners](https://mlsanity.com/paper/2410.02131/) (2024)
- [Sequential Multi-Dimensional Self-Supervised Learning for Clinical Time Series](https://mlsanity.com/paper/2307.10923/) (2023)
- [From Token to Rhythm: A Multi-Scale Approach for ECG-Language Pretraining](https://mlsanity.com/paper/2506.21803/) (2025)
- [Text-to-ECG: 12-Lead Electrocardiogram Synthesis conditioned on Clinical Text Reports](https://mlsanity.com/paper/2303.09395/) (2023)
- [Self-Supervised Pre-Training with Joint-Embedding Predictive Architecture Boosts ECG Classification Performance](https://mlsanity.com/paper/2410.13867/) (2024)
- [Event-Based Contrastive Learning for Medical Time Series](https://mlsanity.com/paper/2312.10308/) (2023)
- [Towards the Practical Utility of Federated Learning in the Medical Domain](https://mlsanity.com/paper/2207.03075/) (2022)
- [ECG-Byte: A Tokenizer for End-to-End Generative Electrocardiogram Language Modeling](https://mlsanity.com/paper/2412.14373/) (2024)
- [BenchECG and xECG: a benchmark and baseline for ECG foundation models](https://mlsanity.com/paper/2509.10151/) (2025)
- [In-Distribution and Out-of-Distribution Self-supervised ECG Representation Learning for Arrhythmia Detection](https://mlsanity.com/paper/2304.06427/) (2023)
- [Masked Transformer for Electrocardiogram Classification](https://mlsanity.com/paper/2309.07136/) (2023)
- [CLEF: Clinically-Guided Contrastive Learning for Electrocardiogram Foundation Models](https://mlsanity.com/paper/2512.02180/) (2025)
- [Electrocardiogram-Language Model for Few-Shot Question Answering with Meta Learning](https://mlsanity.com/paper/2410.14464/) (2024)
- [Knowledge-enhanced Multimodal ECG Representation Learning with Arbitrary-Lead Inputs](https://mlsanity.com/paper/2502.17900/) (2025)
- [Deep Metric Learning for the Hemodynamics Inference with Electrocardiogram Signals](https://mlsanity.com/paper/2308.04650/) (2023)
- [Self-Alignment Learning to Improve Myocardial Infarction Detection from Single-Lead ECG](https://mlsanity.com/paper/2509.19397/) (2025)
- [ECG-Agent: On-Device Tool-Calling Agent for ECG Multi-Turn Dialogue](https://mlsanity.com/paper/2601.20323/) (2026)
- [Pretraining ECG Data with Adversarial Masking Improves Model Generalizability for Data-Scarce Tasks](https://mlsanity.com/paper/2211.07889/) (2022)
- [Signal, Image, or Symbolic: Exploring the Best Input Representation for Electrocardiogram-Language Models Through a Unified Framework](https://mlsanity.com/paper/2505.18847/) (2025)
- [Self-DANA: A Resource-Efficient Channel-Adaptive Self-Supervised Approach for ECG Foundation Models](https://mlsanity.com/paper/2507.14151/) (2025)
- [Which Augmentation Should I Use? An Empirical Investigation of Augmentations for Self-Supervised Phonocardiogram Representation Learning](https://mlsanity.com/paper/2312.00502/) (2023)
- [Tracing the Heart's Pathways: ECG Representation Learning from a Cardiac Conduction Perspective](https://mlsanity.com/paper/2512.24002/) (2025)
- [Domain Knowledge is Power: Leveraging Physiological Priors for Self Supervised Representation Learning in Electrocardiography](https://mlsanity.com/paper/2509.08116/) (2025)
- [ElectroCardioGuard: Preventing Patient Misidentification in Electrocardiogram Databases through Neural Networks](https://mlsanity.com/paper/2306.06196/) (2023)

## Abstract (arXiv)

In recent years, self-supervised learning methods have shown significant improvement for pre-training with unlabeled data and have proven helpful for electrocardiogram signals. However, most previous pre-training methods for electrocardiogram focused on capturing only global contextual representations. This inhibits the models from learning fruitful representation of electrocardiogram, which results in poor performance on downstream tasks. Additionally, they cannot fine-tune the model with an arbitrary set of electrocardiogram leads unless the models were pre-trained on the same set of leads. In this work, we propose an ECG pre-training method that learns both local and global contextual representations for better generalizability and performance on downstream tasks. In addition, we propose random lead masking as an ECG-specific augmentation method to make our proposed model robust to an arbitrary set of leads. Experimental results on two downstream tasks, cardiac arrhythmia classification and patient identification, show that our proposed approach outperforms other state-of-the-art methods.