# Deep evidential fusion with uncertainty quantification and contextual discounting for multimodal medical image segmentation

arXiv: https://arxiv.org/abs/2309.05919 · HTML: https://mlsanity.com/paper/2309.05919/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 109 references that are in the map.
- First posted on arXiv 12 Sep 2023.

## Builds on (most cited first, 27 of 27)

- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges](https://mlsanity.com/paper/2011.06225/) (2020)
- [Evidential Deep Learning to Quantify Classification Uncertainty](https://mlsanity.com/paper/1806.01768/) (2018)
- [A Survey on Visual Transformer](https://mlsanity.com/paper/2012.12556/) (2020)
- [The RSNA-ASNR-MICCAI BraTS 2021 Benchmark on Brain Tumor Segmentation and Radiogenomic Classification](https://mlsanity.com/paper/2107.02314/) (2021)
- [Multimodal Learning with Transformers: A Survey](https://mlsanity.com/paper/2206.06488/) (2022)
- [3D MRI brain tumor segmentation using autoencoder regularization](https://mlsanity.com/paper/1810.11654/) (2018)
- [nnU-Net: Self-adapting Framework for U-Net-Based Medical Image Segmentation](https://mlsanity.com/paper/1809.10486/) (2018)
- [nnFormer: Interleaved Transformer for Volumetric Segmentation](https://mlsanity.com/paper/2109.03201/) (2021)
- [A review: Deep learning for medical image segmentation using multi-modality fusion](https://mlsanity.com/paper/2004.10664/) (2020)
- [Understanding Probabilistic Sparse Gaussian Process Approximations](https://mlsanity.com/paper/1606.04820/) (2016)
- [A Robust Volumetric Transformer for Accurate 3D Tumor Segmentation](https://mlsanity.com/paper/2111.13300/) (2021)
- [Extending nn-UNet for brain tumor segmentation](https://mlsanity.com/paper/2112.04653/) (2021)
- [TBraTS: Trusted Brain Tumor Segmentation](https://mlsanity.com/paper/2206.09309/) (2022)
- [Post Training Uncertainty Calibration of Deep Networks For Medical Image Segmentation](https://mlsanity.com/paper/2010.14290/) (2020)
- [An evidential classifier based on Dempster-Shafer theory and deep learning](https://mlsanity.com/paper/2103.13549/) (2021)
- [Decision-Making with Belief Functions: a Review](https://mlsanity.com/paper/1808.05322/) (2018)
- [Lymphoma segmentation from 3D PET-CT images using a deep evidential network](https://mlsanity.com/paper/2201.13078/) (2022)
- [Evidence fusion with contextual discounting for multi-modality medical image segmentation](https://mlsanity.com/paper/2206.11739/) (2022)
- [Belief function-based semi-supervised learning for brain tumor segmentation](https://mlsanity.com/paper/2102.00097/) (2021)
- [Evidential fully convolutional network for semantic segmentation](https://mlsanity.com/paper/2103.13544/) (2021)
- [A Tri-attention Fusion Guided Multi-modal Segmentation Network](https://mlsanity.com/paper/2111.01623/) (2021)
- [NN-EVCLUS: Neural Network-based Evidential Clustering](https://mlsanity.com/paper/2009.12795/) (2020)
- [Application of belief functions to medical image segmentation: A review](https://mlsanity.com/paper/2205.01733/) (2022)

## Built on by (most cited first, 2 of 2)

- [Has Multimodal Learning Delivered Universal Intelligence in Healthcare? A Comprehensive Survey](https://mlsanity.com/paper/2408.12880/) (2024)
- [Characterizing Text Branch Sensitivity in Medical Vision-Language Segmentation via Evidence Decoupling](https://mlsanity.com/paper/2609.02663/) (2026)

## Abstract (arXiv)

Single-modality medical images generally do not contain enough information to reach an accurate and reliable diagnosis. For this reason, physicians generally diagnose diseases based on multimodal medical images such as, e.g., PET/CT. The effective fusion of multimodal information is essential to reach a reliable decision and explain how the decision is made as well. In this paper, we propose a fusion framework for multimodal medical image segmentation based on deep learning and the Dempster-Shafer theory of evidence. In this framework, the reliability of each single modality image when segmenting different objects is taken into account by a contextual discounting operation. The discounted pieces of evidence from each modality are then combined by Dempster's rule to reach a final decision. Experimental results with a PET-CT dataset with lymphomas and a multi-MRI dataset with brain tumors show that our method outperforms the state-of-the-art methods in accuracy and reliability.