# Analysis of a Deep Learning Model for 12-Lead ECG Classification Reveals Learned Features Similar to Diagnostic Criteria

arXiv: https://arxiv.org/abs/2211.01738 · HTML: https://mlsanity.com/paper/2211.01738/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 55 references that are in the map.
- First posted on arXiv 3 Nov 2022.

## Builds on (most cited first, 16 of 16)

- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [A Survey Of Methods For Explaining Black Box Models](https://mlsanity.com/paper/1802.01933/) (2018)
- [Evaluating the visualization of what a Deep Neural Network has learned](https://mlsanity.com/paper/1509.06321/) (2015)
- [Unmasking Clever Hans Predictors and Assessing What Machines Really Learn](https://mlsanity.com/paper/1902.10178/) (2019)
- [Explaining NonLinear Classification Decisions with Deep Taylor Decomposition](https://mlsanity.com/paper/1512.02479/) (2015)
- [Visualizing Deep Neural Network Decisions: Prediction Difference Analysis](https://mlsanity.com/paper/1702.04595/) (2017)
- [Automatic diagnosis of the 12-lead ECG using a deep neural network](https://mlsanity.com/paper/1904.01949/) (2019)
- [Towards Best Practice in Explaining Neural Network Decisions with LRP](https://mlsanity.com/paper/1910.09840/) (2019)
- [Leakage and the Reproducibility Crisis in ML-based Science](https://mlsanity.com/paper/2207.07048/) (2022)
- [iNNvestigate neural networks!](https://mlsanity.com/paper/1808.04260/) (2018)
- [Explainable artificial intelligence model to predict acute critical illness from electronic health records](https://mlsanity.com/paper/1912.01266/) (2019)
- [Interpretable Deep Neural Networks for Single-Trial EEG Classification](https://mlsanity.com/paper/1604.08201/) (2016)
- [Understanding and Comparing Deep Neural Networks for Age and Gender Classification](https://mlsanity.com/paper/1708.07689/) (2017)
- [ECG Beat Representation and Delineation by means of Variable Projection](https://mlsanity.com/paper/2109.13022/) (2021)

## Built on by (most cited first, 11 of 11)

- [Explaining Deep Learning for ECG Analysis: Building Blocks for Auditing and Knowledge Discovery](https://mlsanity.com/paper/2305.17043/) (2023)
- [Using explainable AI to investigate electrocardiogram changes during healthy aging -- from expert features to raw signals](https://mlsanity.com/paper/2310.07463/) (2023)
- [TransECG: Leveraging Transformers for Explainable ECG Re-identification Risk Analysis](https://mlsanity.com/paper/2503.13495/) (2025)
- [Learning ECG Signal Features Without Backpropagation Using Linear Laws](https://mlsanity.com/paper/2307.01930/) (2023)
- [Rational Gaussian wavelets and corresponding model driven neural networks](https://mlsanity.com/paper/2502.01282/) (2025)
- [ArrhythmiaVision: Resource-Conscious Deep Learning Models with Visual Explanations for ECG Arrhythmia Classification](https://mlsanity.com/paper/2505.03787/) (2025)
- [Benchmarking the Impact of Noise on Deep Learning-based Classification of Atrial Fibrillation in 12-Lead ECG](https://mlsanity.com/paper/2303.13915/) (2023)
- [The Role of Explainable AI in Revolutionizing Human Health Monitoring: A Review](https://mlsanity.com/paper/2409.07347/) (2024)
- [EXGnet: a single-lead explainable-AI guided multiresolution network with train-only quantitative features for trustworthy ECG arrhythmia classification](https://mlsanity.com/paper/2506.12404/) (2025)
- [Shift-Invariant Feature Attribution in the Application of Wireless Electrocardiograms](https://mlsanity.com/paper/2603.20462/) (2026)
- [Beyond Local Inspection: Global, Guideline-Grounded Evaluation of Post-hoc XAI Methods for ECG Classification](https://mlsanity.com/paper/2607.24035/) (2026)

## Abstract (arXiv)

Despite their remarkable performance, deep neural networks remain unadopted in clinical practice, which is considered to be partially due to their lack in explainability. In this work, we apply attribution methods to a pre-trained deep neural network (DNN) for 12-lead electrocardiography classification to open this "black box" and understand the relationship between model prediction and learned features. We classify data from a public data set and the attribution methods assign a "relevance score" to each sample of the classified signals. This allows analyzing what the network learned during training, for which we propose quantitative methods: average relevance scores over a) classes, b) leads, and c) average beats. The analyses of relevance scores for atrial fibrillation (AF) and left bundle branch block (LBBB) compared to healthy controls show that their mean values a) increase with higher classification probability and correspond to false classifications when around zero, and b) correspond to clinical recommendations regarding which lead to consider. Furthermore, c) visible P-waves and concordant T-waves result in clearly negative relevance scores in AF and LBBB classification, respectively. In summary, our analysis suggests that the DNN learned features similar to cardiology textbook knowledge.