# Explainable Deep Neural Network for Multimodal ECG Signals: Intermediate vs Late Fusion

arXiv: https://arxiv.org/abs/2508.11666 · HTML: https://mlsanity.com/paper/2508.11666/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 102 references that are in the map.
- First posted on arXiv 6 Aug 2025.

## Builds on (most cited first, 15 of 15)

- [Sanity Checks for Saliency Maps](https://mlsanity.com/paper/1810.03292/) (2018)
- [Multimodal Machine Learning: A Survey and Taxonomy](https://mlsanity.com/paper/1705.09406/) (2017)
- [A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAI](https://mlsanity.com/paper/1907.07374/) (2019)
- [Deep Learning for ECG Analysis: Benchmarks and Insights from PTB-XL](https://mlsanity.com/paper/2004.13701/) (2020)
- [A Review of Feature Selection Methods Based on Mutual Information](https://mlsanity.com/paper/1509.07577/) (2015)
- [Cardiologist-Level Arrhythmia Detection with Convolutional Neural Networks](https://mlsanity.com/paper/1707.01836/) (2017)
- [Quantifying Interpretability and Trust in Machine Learning Systems](https://mlsanity.com/paper/1901.08558/) (2019)
- [Rethinking Multimodality: Optimizing Multimodal Deep Learning for Biomedical Signal Classification](https://mlsanity.com/paper/2508.00963/) (2025)
- [MMLF: Multi-modal Multi-class Late Fusion for Object Detection with Uncertainty Estimation](https://mlsanity.com/paper/2410.08739/) (2024)
- [Robust heartbeat detection using multimodal recordings and ECG quality assessment with signal amplitudes dispersion](https://mlsanity.com/paper/2105.10046/) (2021)
- [MVMTnet: A Multi-variate Multi-modal Transformer for Multi-class Classification of Cardiac Irregularities Using ECG Waveforms and Clinical Notes](https://mlsanity.com/paper/2302.11021/) (2023)
- [A Study of Deep Feature Fusion based Methods for Classifying Multi-lead ECG](https://mlsanity.com/paper/1808.01721/) (2018)
- [DeepMI: Deep Multi-lead ECG Fusion for Identifying Myocardial Infarction and its Occurrence-time](https://mlsanity.com/paper/2104.02054/) (2021)
- [Deep Learning Models for Arrhythmia Classification Using Stacked Time-frequency Scalogram Images from ECG Signals](https://mlsanity.com/paper/2312.09426/) (2023)
- [Arrhythmia Classification from 12-Lead ECG Signals Using Convolutional and Transformer-Based Deep Learning Models](https://mlsanity.com/paper/2502.17887/) (2025)

## Built on by (most cited first, 6 of 6)

- [When Should a Model Change Its Mind? An Energy-Based Theory and Regularizer for Concept Drift in Electrocardiogram (ECG) Signals](https://mlsanity.com/paper/2602.22294/) (2026)
- [Complementarity-Preserving Generative Theory for Multimodal ECG Synthesis: A Quantum-Inspired Approach](https://mlsanity.com/paper/2603.26695/) (2026)
- [Attractor-Vascular Coupling Theory: Formal Grounding and Empirical Validation for AAMI-Standard Cuffless Blood Pressure Estimation from Smartphone Photoplethysmography](https://mlsanity.com/paper/2605.10871/) (2026)
- [A Nonlinear Complexity Index for Wearable PPG Cardiovascular Stability: Multiscale Validation, Systematic Evaluation Correction, and Bayesian Parameter Optimization](https://mlsanity.com/paper/2605.18802/) (2026)
- [Attractor Domain Theory: A Mathematical Framework for Cardiovascular Attractor Analysis with Wearable Photoplethysmography (PPG) Validation](https://mlsanity.com/paper/2606.22039/) (2026)
- [CFD-Guided Detection of Concept Drift in Multimodal Physiologic Signals](https://mlsanity.com/paper/2608.07759/) (2026)

## Abstract (arXiv)

The limitations of unimodal deep learning models, particularly their tendency to overfit and limited generalizability, have renewed interest in multimodal fusion strategies. Multimodal deep neural networks (MDNN) have the capability of integrating diverse data domains and offer a promising solution for robust and accurate predictions. However, the optimal fusion strategy, intermediate fusion (feature-level) versus late fusion (decision-level) remains insufficiently examined, especially in high-stakes clinical contexts such as ECG-based cardiovascular disease (CVD) classification. This study investigates the comparative effectiveness of intermediate and late fusion strategies using ECG signals across three domains: time, frequency, and time-frequency. A series of experiments were conducted to identify the highest-performing fusion architecture. Results demonstrate that intermediate fusion consistently outperformed late fusion, achieving a peak accuracy of 97 percent, with Cohen's d > 0.8 relative to standalone models and d = 0.40 compared to late fusion. Interpretability analyses using saliency maps reveal that both models align with the discretized ECG signals. Statistical dependency between the discretized ECG signals and corresponding saliency maps for each class was confirmed using Mutual Information (MI). The proposed ECG domain-based multimodal model offers superior predictive capability and enhanced explainability, crucial attributes in medical AI applications, surpassing state-of-the-art models.