# ECG Heartbeat Classification Using Multimodal Fusion

arXiv: https://arxiv.org/abs/2107.09869 · HTML: https://mlsanity.com/paper/2107.09869/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 74 references that are in the map.
- First posted on arXiv 21 Jul 2021.

## Builds on (most cited first, 5 of 5)

- [Imaging Time-Series to Improve Classification and Imputation](https://mlsanity.com/paper/1506.00327/) (2015)
- [ECG Heartbeat Classification: A Deep Transferable Representation](https://mlsanity.com/paper/1805.00794/) (2018)
- [CNN based Multistage Gated Average Fusion (MGAF) for Human Action Recognition Using Depth and Inertial Sensors](https://mlsanity.com/paper/2010.16073/) (2020)
- [Multi-level Stress Assessment Using Multi-domain Fusion of ECG Signal](https://mlsanity.com/paper/2008.05503/) (2020)
- [Towards Improved Human Action Recognition Using Convolutional Neural Networks and Multimodal Fusion of Depth and Inertial Sensor Data](https://mlsanity.com/paper/2008.09747/) (2020)

## Built on by (most cited first, 10 of 10)

- [Synthetic ECG Signal Generation using Probabilistic Diffusion Models](https://mlsanity.com/paper/2303.02475/) (2023)
- [Imbalanced Big Data Oversampling: Taxonomy, Algorithms, Software, Guidelines and Future Directions](https://mlsanity.com/paper/2107.11508/) (2021)
- [A Survey on Physiological Signal Based Emotion Recognition](https://mlsanity.com/paper/2205.10466/) (2022)
- [Rethinking Multimodality: Optimizing Multimodal Deep Learning for Biomedical Signal Classification](https://mlsanity.com/paper/2508.00963/) (2025)
- [HeartBERT: A Self-Supervised ECG Embedding Model for Efficient and Effective Medical Signal Analysis](https://mlsanity.com/paper/2411.11896/) (2024)
- [MVMTnet: A Multi-variate Multi-modal Transformer for Multi-class Classification of Cardiac Irregularities Using ECG Waveforms and Clinical Notes](https://mlsanity.com/paper/2302.11021/) (2023)
- [Multimodal Classification: Current Landscape, Taxonomy and Future Directions](https://mlsanity.com/paper/2109.09020/) (2021)
- [rECGnition_v2.0: Self-Attentive Canonical Fusion of ECG and Patient Data using deep learning for effective Cardiac Diagnostics](https://mlsanity.com/paper/2502.16255/) (2025)
- [Heart Rate Classification in ECG Signals Using Machine Learning and Deep Learning](https://mlsanity.com/paper/2506.06349/) (2025)
- [Phase-OTDR Event Detection Using Image-Based Data Transformation and Deep Learning](https://mlsanity.com/paper/2512.05830/) (2025)

## Abstract (arXiv)

Electrocardiogram (ECG) is an authoritative source to diagnose and counter critical cardiovascular syndromes such as arrhythmia and myocardial infarction (MI). Current machine learning techniques either depend on manually extracted features or large and complex deep learning networks which merely utilize the 1D ECG signal directly. Since intelligent multimodal fusion can perform at the stateof-the-art level with an efficient deep network, therefore, in this paper, we propose two computationally efficient multimodal fusion frameworks for ECG heart beat classification called Multimodal Image Fusion (MIF) and Multimodal Feature Fusion (MFF). At the input of these frameworks, we convert the raw ECG data into three different images using Gramian Angular Field (GAF), Recurrence Plot (RP) and Markov Transition Field (MTF). In MIF, we first perform image fusion by combining three imaging modalities to create a single image modality which serves as input to the Convolutional Neural Network (CNN). In MFF, we extracted features from penultimate layer of CNNs and fused them to get unique and interdependent information necessary for better performance of classifier. These informational features are finally used to train a Support Vector Machine (SVM) classifier for ECG heart-beat classification. We demonstrate the superiority of the proposed fusion models by performing experiments on PhysioNets MIT-BIH dataset for five distinct conditions of arrhythmias which are consistent with the AAMI EC57 protocols and on PTB diagnostics dataset for Myocardial Infarction (MI) classification. We achieved classification accuracy of 99.7% and 99.2% on arrhythmia and MI classification, respectively.