# Evidence-aware multi-modal data fusion and its application to total knee replacement prediction

arXiv: https://arxiv.org/abs/2303.13810 · HTML: https://mlsanity.com/paper/2303.13810/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 46 references that are in the map.
- First posted on arXiv 24 Mar 2023.

## Builds on (most cited first, 13 of 13)

- [A review of deep learning in medical imaging: Imaging traits, technology trends, case studies with progress highlights, and future promises](https://mlsanity.com/paper/2008.09104/) (2020)
- [Self-supervised Feature Learning via Exploiting Multi-modal Data for Retinal Disease Diagnosis](https://mlsanity.com/paper/2007.11067/) (2020)
- [Multi-modal Graph Learning for Disease Prediction](https://mlsanity.com/paper/2203.05880/) (2022)
- [Multimodal Machine Learning-based Knee Osteoarthritis Progression Prediction from Plain Radiographs and Clinical Data](https://mlsanity.com/paper/1904.06236/) (2019)
- [Causal Hidden Markov Model for Time Series Disease Forecasting](https://mlsanity.com/paper/2103.16391/) (2021)
- [Brain Age Estimation From MRI Using Cascade Networks with Ranking Loss](https://mlsanity.com/paper/2106.03052/) (2021)
- [Evidence fusion with contextual discounting for multi-modality medical image segmentation](https://mlsanity.com/paper/2206.11739/) (2022)
- [Deep Multimodal Guidance for Medical Image Classification](https://mlsanity.com/paper/2203.05683/) (2022)
- [Deep learning predicts total knee replacement from magnetic resonance images](https://mlsanity.com/paper/2002.10591/) (2020)
- [Clinically-Inspired Multi-Agent Transformers for Disease Trajectory Forecasting from Multimodal Data](https://mlsanity.com/paper/2210.13889/) (2022)
- [MRI-based Alzheimer's disease prediction via distilling the knowledge in multi-modal data](https://mlsanity.com/paper/2104.03618/) (2021)
- [Covid-19 classification with deep neural network and belief functions](https://mlsanity.com/paper/2101.06958/) (2021)
- [BIO-CXRNET: A Robust Multimodal Stacking Machine Learning Technique for Mortality Risk Prediction of COVID-19 Patients using Chest X-Ray Images and Clinical Data](https://mlsanity.com/paper/2206.07595/) (2022)

## Built on by (most cited first, 1 of 1)

- [A review of uncertainty quantification in medical image analysis: probabilistic and non-probabilistic methods](https://mlsanity.com/paper/2310.06873/) (2023)

## Abstract (arXiv)

Deep neural networks have been widely studied for predicting a medical condition, such as total knee replacement (TKR). It has shown that data of different modalities, such as imaging data, clinical variables and demographic information, provide complementary information and thus can improve the prediction accuracy together. However, the data sources of various modalities may not always be of high quality, and each modality may have only partial information of medical condition. Thus, predictions from different modalities can be opposite, and the final prediction may fail in the presence of such a conflict. Therefore, it is important to consider the reliability of each source data and the prediction output when making a final decision. In this paper, we propose an evidence-aware multi-modal data fusion framework based on the Dempster-Shafer theory (DST). The backbone models contain an image branch, a non-image branch and a fusion branch. For each branch, there is an evidence network that takes the extracted features as input and outputs an evidence score, which is designed to represent the reliability of the output from the current branch. The output probabilities along with the evidence scores from multiple branches are combined with the Dempster's combination rule to make a final prediction. Experimental results on the public OA initiative (OAI) dataset for the TKR prediction task show the superiority of the proposed fusion strategy on various backbone models.