# Interpretable Multimodal Learning for Cardiovascular Hemodynamics Assessment

arXiv: https://arxiv.org/abs/2404.04718 · HTML: https://mlsanity.com/paper/2404.04718/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 59 references that are in the map.
- First posted on arXiv 6 Apr 2024.

## Builds on (most cited first, 14 of 14)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Fast Graph Representation Learning with PyTorch Geometric](https://mlsanity.com/paper/1903.02428/) (2019)
- [Learning Spatio-Temporal Features with 3D Residual Networks for Action Recognition](https://mlsanity.com/paper/1708.07632/) (2017)
- [Integrated multimodal artificial intelligence framework for healthcare applications](https://mlsanity.com/paper/2202.12998/) (2022)
- [A Combined Deep-Learning and Deformable-Model Approach to Fully Automatic Segmentation of the Left Ventricle in Cardiac MRI](https://mlsanity.com/paper/1512.07951/) (2015)
- [Interpretable Deep Models for Cardiac Resynchronisation Therapy Response Prediction](https://mlsanity.com/paper/2006.13811/) (2020)
- [Visual Explanations From Deep 3D Convolutional Neural Networks for Alzheimer's Disease Classification](https://mlsanity.com/paper/1803.02544/) (2018)
- [Understanding Neural Networks and Individual Neuron Importance via Information-Ordered Cumulative Ablation](https://mlsanity.com/paper/1804.06679/) (2018)
- [Uncertainty Estimation for Heatmap-based Landmark Localization](https://mlsanity.com/paper/2203.02351/) (2022)
- [Automatic quantification of the LV function and mass: a deep learning approach for cardiovascular MRI](https://mlsanity.com/paper/1812.06061/) (2018)
- [CNN-based Landmark Detection in Cardiac CTA Scans](https://mlsanity.com/paper/1804.04963/) (2018)
- [Tensor-based Multimodal Learning for Prediction of Pulmonary Arterial Wedge Pressure from Cardiac MRI](https://mlsanity.com/paper/2303.07540/) (2023)
- [PyKale: Knowledge-Aware Machine Learning from Multiple Sources in Python](https://mlsanity.com/paper/2106.09756/) (2021)

## Built on by (most cited first, 2 of 2)

- [Towards Cardiac MRI Foundation Models: Comprehensive Visual-Tabular Representations for Whole-Heart Assessment and Beyond](https://mlsanity.com/paper/2504.13037/) (2025)
- [Multimodal Latent Fusion of ECG Leads for Early Assessment of Pulmonary Hypertension](https://mlsanity.com/paper/2503.13470/) (2025)

## Abstract (arXiv)

Pulmonary Arterial Wedge Pressure (PAWP) is an essential cardiovascular hemodynamics marker to detect heart failure. In clinical practice, Right Heart Catheterization is considered a gold standard for assessing cardiac hemodynamics while non-invasive methods are often needed to screen high-risk patients from a large population. In this paper, we propose a multimodal learning pipeline to predict PAWP marker. We utilize complementary information from Cardiac Magnetic Resonance Imaging (CMR) scans (short-axis and four-chamber) and Electronic Health Records (EHRs). We extract spatio-temporal features from CMR scans using tensor-based learning. We propose a graph attention network to select important EHR features for prediction, where we model subjects as graph nodes and feature relationships as graph edges using the attention mechanism. We design four feature fusion strategies: early, intermediate, late, and hybrid fusion. With a linear classifier and linear fusion strategies, our pipeline is interpretable. We validate our pipeline on a large dataset of 2,641 subjects from our ASPIRE registry. The comparative study against state-of-the-art methods confirms the superiority of our pipeline. The decision curve analysis further validates that our pipeline can be applied to screen a large population. The code is available at https://github.com/prasunc/hemodynamics.