# Simultaneous Tri-Modal Medical Image Fusion and Super-Resolution using Conditional Diffusion Model

arXiv: https://arxiv.org/abs/2404.17357 · HTML: https://mlsanity.com/paper/2404.17357/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models.
- Builds on 12 of its 37 references that are in the map.
- First posted on arXiv 26 Apr 2024.

## Placed in

- Diffusion models / [Medical imaging and segmentation](https://mlsanity.com/diffusion/medical-imaging-and-segmentation/)

## Builds on (most cited first, 12 of 12)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [Image Super-Resolution via Iterative Refinement](https://mlsanity.com/paper/2104.07636/) (2021)
- [CDDFuse: Correlation-Driven Dual-Branch Feature Decomposition for Multi-Modality Image Fusion](https://mlsanity.com/paper/2211.14461/) (2022)
- [DDFM: Denoising Diffusion Model for Multi-Modality Image Fusion](https://mlsanity.com/paper/2303.06840/) (2023)
- [Dif-Fusion: Towards High Color Fidelity in Infrared and Visible Image Fusion with Diffusion Models](https://mlsanity.com/paper/2301.08072/) (2023)
- [TGFuse: An Infrared and Visible Image Fusion Approach Based on Transformer and Generative Adversarial Network](https://mlsanity.com/paper/2201.10147/) (2022)
- [DisC-Diff: Disentangled Conditional Diffusion Model for Multi-Contrast MRI Super-Resolution](https://mlsanity.com/paper/2303.13933/) (2023)

## Built on by (most cited first, 13 of 13)

- [Shuffle Mamba: State Space Models with Random Shuffle for Multi-Modal Image Fusion](https://mlsanity.com/paper/2409.01728/) (2024)
- [FlexiD-Fuse: Flexible number of inputs multi-modal medical image fusion based on diffusion model](https://mlsanity.com/paper/2509.09456/) (2025)
- [Volumetric Conditioning Module to Control Pretrained Diffusion Models for 3D Medical Images](https://mlsanity.com/paper/2410.21826/) (2024)
- [MambaTrans: Multimodal Fusion Image Translation via Large Language Model Priors for Downstream Visual Tasks](https://mlsanity.com/paper/2508.07803/) (2025)
- [Text-Guided Channel Perturbation and Pretrained Knowledge Integration for Unified Multi-Modality Image Fusion](https://mlsanity.com/paper/2511.12432/) (2025)
- [OmniFM: Toward Modality-Robust and Task-Agnostic Federated Learning for Heterogeneous Medical Imaging](https://mlsanity.com/paper/2603.21660/) (2026)
- [Fuse4Seg: Image Fusion for Multi-Modal Medical Segmentation via Bi-level Optimization](https://mlsanity.com/paper/2409.10328/) (2024)
- [CAWM-Mamba: A unified model for infrared-visible image fusion and compound adverse weather restoration](https://mlsanity.com/paper/2603.02560/) (2026)
- [TriFusion-SR: Joint Tri-Modal Medical Image Fusion and SR](https://mlsanity.com/paper/2603.09702/) (2026)
- [Cone-Beam CT Image Quality Enhancement Using A Latent Diffusion Model Trained with Simulated CBCT Artifacts](https://mlsanity.com/paper/2603.26014/) (2026)
- [Degradation-Robust Fusion: An Efficient Degradation-Aware Diffusion Framework for Multimodal Image Fusion in Arbitrary Degradation Scenarios](https://mlsanity.com/paper/2604.08922/) (2026)
- [Degradation-Aware Blur-Segmentation of Brain Tumor](https://mlsanity.com/paper/2605.15671/) (2026)
- [Multi-modality Image Fusion under Adverse Weather: Mask-Guided Feature Restoration and Interaction](https://mlsanity.com/paper/2606.26812/) (2026)

## Abstract (arXiv)

In clinical practice, tri-modal medical image fusion, compared to the existing dual-modal technique, can provide a more comprehensive view of the lesions, aiding physicians in evaluating the disease's shape, location, and biological activity. However, due to the limitations of imaging equipment and considerations for patient safety, the quality of medical images is usually limited, leading to sub-optimal fusion performance, and affecting the depth of image analysis by the physician. Thus, there is an urgent need for a technology that can both enhance image resolution and integrate multi-modal information. Although current image processing methods can effectively address image fusion and super-resolution individually, solving both problems synchronously remains extremely challenging. In this paper, we propose TFS-Diff, a simultaneously realize tri-modal medical image fusion and super-resolution model. Specially, TFS-Diff is based on the diffusion model generation of a random iterative denoising process. We also develop a simple objective function and the proposed fusion super-resolution loss, effectively evaluates the uncertainty in the fusion and ensures the stability of the optimization process. And the channel attention module is proposed to effectively integrate key information from different modalities for clinical diagnosis, avoiding information loss caused by multiple image processing. Extensive experiments on public Harvard datasets show that TFS-Diff significantly surpass the existing state-of-the-art methods in both quantitative and visual evaluations. Code is available at https://github.com/XylonXu01/TFS-Diff.