# Multimodality Multi-Lead ECG Arrhythmia Classification using Self-Supervised Learning

arXiv: https://arxiv.org/abs/2210.06297 · HTML: https://mlsanity.com/paper/2210.06297/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 38 references that are in the map.
- First posted on arXiv 30 Sep 2022.

## Builds on (most cited first, 18 of 18)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Show, Attend and Tell: Neural Image Caption Generation with Visual Attention](https://mlsanity.com/paper/1502.03044/) (2015)
- [Colorful Image Colorization](https://mlsanity.com/paper/1603.08511/) (2016)
- [Unsupervised Learning of Video Representations using LSTMs](https://mlsanity.com/paper/1502.04681/) (2015)
- [Bag of Tricks for Image Classification with Convolutional Neural Networks](https://mlsanity.com/paper/1812.01187/) (2018)
- [CLOCS: Contrastive Learning of Cardiac Signals Across Space, Time, and Patients](https://mlsanity.com/paper/2005.13249/) (2020)
- [Automatic diagnosis of the 12-lead ECG using a deep neural network](https://mlsanity.com/paper/1904.01949/) (2019)
- [Learning Features by Watching Objects Move](https://mlsanity.com/paper/1612.06370/) (2016)
- [Self-supervised representation learning from 12-lead ECG data](https://mlsanity.com/paper/2103.12676/) (2021)
- [Cardiologist-Level Arrhythmia Detection with Convolutional Neural Networks](https://mlsanity.com/paper/1707.01836/) (2017)
- [Deep Reinforcement Learning in Computer Vision: A Comprehensive Survey](https://mlsanity.com/paper/2108.11510/) (2021)
- [Intra-Inter Subject Self-supervised Learning for Multivariate Cardiac Signals](https://mlsanity.com/paper/2109.08908/) (2021)
- [Cross and Learn: Cross-Modal Self-Supervision](https://mlsanity.com/paper/1811.03879/) (2018)
- [Deep reinforcement learning in medical imaging: A literature review](https://mlsanity.com/paper/2103.05115/) (2021)
- [SS-3DCapsNet: Self-supervised 3D Capsule Networks for Medical Segmentation on Less Labeled Data](https://mlsanity.com/paper/2201.05905/) (2022)
- [Self-Supervised Learning via multi-Transformation Classification for Action Recognition](https://mlsanity.com/paper/2102.10378/) (2021)

## Built on by (most cited first, 3 of 3)

- [TSRNet: Simple Framework for Real-time ECG Anomaly Detection with Multimodal Time and Spectrogram Restoration Network](https://mlsanity.com/paper/2312.10187/) (2023)
- [FADE: Forecasting for Anomaly Detection on ECG](https://mlsanity.com/paper/2502.07389/) (2025)
- [CuPID: Leveraging Masked Single-Lead ECG Modelling for Enhancing the Representations](https://mlsanity.com/paper/2502.21127/) (2025)

## Abstract (arXiv)

Electrocardiogram (ECG) signal is one of the most effective sources of information mainly employed for the diagnosis and prediction of cardiovascular diseases (CVDs) connected with the abnormalities in heart rhythm. Clearly, single modality ECG (i.e. time series) cannot convey its complete characteristics, thus, exploiting both time and time-frequency modalities in the form of time-series data and spectrogram is needed. Leveraging the cutting-edge self-supervised learning (SSL) technique on unlabeled data, we propose SSL-based multimodality ECG classification. Our proposed network follows SSL learning paradigm and consists of two modules corresponding to pre-stream task, and down-stream task, respectively. In the SSL-pre-stream task, we utilize self-knowledge distillation (KD) techniques with no labeled data, on various transformations and in both time and frequency domains. In the down-stream task, which is trained on labeled data, we propose a gate fusion mechanism to fuse information from multimodality.To evaluate the effectiveness of our approach, ten-fold cross validation on the 12-lead PhysioNet 2020 dataset has been conducted.