# CNN-LSTM Based Multimodal MRI and Clinical Data Fusion for Predicting Functional Outcome in Stroke Patients

arXiv: https://arxiv.org/abs/2205.05545 · HTML: https://mlsanity.com/paper/2205.05545/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 25 references that are in the map.
- First posted on arXiv 11 May 2022.

## Builds on (most cited first, 4 of 4)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Improving LSTM-based Video Description with Linguistic Knowledge Mined from Text](https://mlsanity.com/paper/1604.01729/) (2016)

## Built on by (most cited first, 6 of 6)

- [ISLES'24: Final Infarct Prediction with Multimodal Imaging and Clinical Data. Where Do We Stand?](https://mlsanity.com/paper/2408.10966/) (2024)
- [MVMTnet: A Multi-variate Multi-modal Transformer for Multi-class Classification of Cardiac Irregularities Using ECG Waveforms and Clinical Notes](https://mlsanity.com/paper/2302.11021/) (2023)
- [Automatic Prediction of Stroke Treatment Outcomes: Latest Advances and Perspectives](https://mlsanity.com/paper/2412.04812/) (2024)
- [A Novel Autoencoders-LSTM Model for Stroke Outcome Prediction using Multimodal MRI Data](https://mlsanity.com/paper/2303.09484/) (2023)
- [Fusion of Diffusion Weighted MRI and Clinical Data for Predicting Functional Outcome after Acute Ischemic Stroke with Deep Contrastive Learning](https://mlsanity.com/paper/2402.10894/) (2024)
- [Challenges and proposed solutions in modeling multimodal medical data: A systematic review](https://mlsanity.com/paper/2505.06945/) (2025)

## Abstract (arXiv)

Clinical outcome prediction plays an important role in stroke patient management. From a machine learning point-of-view, one of the main challenges is dealing with heterogeneous data at patient admission, i.e. the image data which are multidimensional and the clinical data which are scalars. In this paper, a multimodal convolutional neural network - long short-term memory (CNN-LSTM) based ensemble model is proposed. For each MR image module, a dedicated network provides preliminary prediction of the clinical outcome using the modified Rankin scale (mRS). The final mRS score is obtained by merging the preliminary probabilities of each module dedicated to a specific type of MR image weighted by the clinical metadata, here age or the National Institutes of Health Stroke Scale (NIHSS). The experimental results demonstrate that the proposed model surpasses the baselines and offers an original way to automatically encode the spatio-temporal context of MR images in a deep learning architecture. The highest AUC (0.77) was achieved for the proposed model with NIHSS.