# MSW-Transformer: Multi-Scale Shifted Windows Transformer Networks for 12-Lead ECG Classification

arXiv: https://arxiv.org/abs/2306.12098 · HTML: https://mlsanity.com/paper/2306.12098/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 46 references that are in the map.
- First posted on arXiv 21 Jun 2023.

## Builds on (most cited first, 13 of 13)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Transformer Interpretability Beyond Attention Visualization](https://mlsanity.com/paper/2012.09838/) (2020)
- [Gated Transformer Networks for Multivariate Time Series Classification](https://mlsanity.com/paper/2103.14438/) (2021)
- [Cardiologist-Level Arrhythmia Detection with Convolutional Neural Networks](https://mlsanity.com/paper/1707.01836/) (2017)
- [Classification of 12-Lead ECG Signals with Bi-directional LSTM Network](https://mlsanity.com/paper/1811.02090/) (2018)
- [Interpreting Deep Neural Networks for Single-Lead ECG Arrhythmia Classification](https://mlsanity.com/paper/2004.05399/) (2020)
- [Beat by Beat: Classifying Cardiac Arrhythmias with Recurrent Neural Networks](https://mlsanity.com/paper/1710.06319/) (2017)
- [ECGNET: Learning where to attend for detection of atrial fibrillation with deep visual attention](https://mlsanity.com/paper/1812.07422/) (2018)
- [IMLE-Net: An Interpretable Multi-level Multi-channel Model for ECG Classification](https://mlsanity.com/paper/2204.05116/) (2022)

## Built on by (most cited first, 4 of 4)

- [ECGMamba: Towards Efficient ECG Classification with BiSSM](https://mlsanity.com/paper/2406.10098/) (2024)
- [ECG-Expert-QA: A Benchmark for Evaluating Medical Large Language Models in Heart Disease Diagnosis](https://mlsanity.com/paper/2502.17475/) (2025)
- [ECG-NAT: A Self-supervised Neighborhood Attention Transformer for Multi-lead Electrocardiogram Classification](https://mlsanity.com/paper/2605.13194/) (2026)
- [MSAIC-Net: A Multi-Scale Attention and Imbalance-Aware Contrastive Network for ECG-Based Myocardial Substrate Abnormality Detection](https://mlsanity.com/paper/2606.06718/) (2026)

## Abstract (arXiv)

Automatic classification of electrocardiogram (ECG) signals plays a crucial role in the early prevention and diagnosis of cardiovascular diseases. While ECG signals can be used for the diagnosis of various diseases, their pathological characteristics exhibit minimal variations, posing a challenge to automatic classification models. Existing methods primarily utilize convolutional neural networks to extract ECG signal features for classification, which may not fully capture the pathological feature differences of different diseases. Transformer networks have advantages in feature extraction for sequence data, but the complete network is complex and relies on large-scale datasets. To address these challenges, we propose a single-layer Transformer network called Multi-Scale Shifted Windows Transformer Networks (MSW-Transformer), which uses a multi-window sliding attention mechanism at different scales to capture features in different dimensions. The self-attention is restricted to non-overlapping local windows via shifted windows, and different window scales have different receptive fields. A learnable feature fusion method is then proposed to integrate features from different windows to further enhance model performance. Furthermore, we visualize the attention mechanism of the multi-window shifted mechanism to achieve better clinical interpretation in the ECG classification task. The proposed model achieves state-of-the-art performance on five classification tasks of the PTBXL-2020 12-lead ECG dataset, which includes 5 diagnostic superclasses, 23 diagnostic subclasses, 12 rhythm classes, 17 morphology classes, and 44 diagnosis classes, with average macro-F1 scores of 77.85%, 47.57%, 66.13%, 34.60%, and 34.29%, and average sample-F1 scores of 81.26%, 68.27%, 91.32%, 50.07%, and 63.19%, respectively.