# MCAD: Multi-modal Conditioned Adversarial Diffusion Model for High-Quality PET Image Reconstruction

arXiv: https://arxiv.org/abs/2406.13150 · HTML: https://mlsanity.com/paper/2406.13150/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models.
- Builds on 12 of its 29 references that are in the map.
- First posted on arXiv 19 Jun 2024.

## Placed in

- Diffusion models / [Medical imaging and segmentation](https://mlsanity.com/diffusion/medical-imaging-and-segmentation/)

## Builds on (most cited first, 12 of 12)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Image Super-Resolution via Iterative Refinement](https://mlsanity.com/paper/2104.07636/) (2021)
- [Tackling the Generative Learning Trilemma with Denoising Diffusion GANs](https://mlsanity.com/paper/2112.07804/) (2021)
- [Making the Most of Text Semantics to Improve Biomedical Vision--Language Processing](https://mlsanity.com/paper/2204.09817/) (2022)
- [Unsupervised Medical Image Translation with Adversarial Diffusion Models](https://mlsanity.com/paper/2207.08208/) (2022)
- [CoLa-Diff: Conditional Latent Diffusion Model for Multi-Modal MRI Synthesis](https://mlsanity.com/paper/2303.14081/) (2023)
- [PET image denoising based on denoising diffusion probabilistic models](https://mlsanity.com/paper/2209.06167/) (2022)
- [Contrastive Diffusion Model with Auxiliary Guidance for Coarse-to-Fine PET Reconstruction](https://mlsanity.com/paper/2308.10157/) (2023)
- [3D Segmentation Guided Style-based Generative Adversarial Networks for PET Synthesis](https://mlsanity.com/paper/2205.08887/) (2022)
- [Full-count PET Recovery from Low-count Image Using a Dilated Convolutional Neural Network](https://mlsanity.com/paper/1910.11865/) (2019)
- [TriDo-Former: A Triple-Domain Transformer for Direct PET Reconstruction from Low-Dose Sinograms](https://mlsanity.com/paper/2308.05365/) (2023)

## Built on by (most cited first, 2 of 2)

- [M2Diff: Multi-Modality Multi-Task Enhanced Diffusion Model for MRI-Guided Low-Dose PET Enhancement](https://mlsanity.com/paper/2603.09075/) (2026)
- [FlowPET: Physics-Informed Symplectic Flow Matching for Low-Count PET Reconstruction](https://mlsanity.com/paper/2607.11104/) (2026)

## Abstract (arXiv)

Radiation hazards associated with standard-dose positron emission tomography (SPET) images remain a concern, whereas the quality of low-dose PET (LPET) images fails to meet clinical requirements. Therefore, there is great interest in reconstructing SPET images from LPET images. However, prior studies focus solely on image data, neglecting vital complementary information from other modalities, e.g., patients' clinical tabular, resulting in compromised reconstruction with limited diagnostic utility. Moreover, they often overlook the semantic consistency between real SPET and reconstructed images, leading to distorted semantic contexts. To tackle these problems, we propose a novel Multi-modal Conditioned Adversarial Diffusion model (MCAD) to reconstruct SPET images from multi-modal inputs, including LPET images and clinical tabular. Specifically, our MCAD incorporates a Multi-modal conditional Encoder (Mc-Encoder) to extract multi-modal features, followed by a conditional diffusion process to blend noise with multi-modal features and gradually map blended features to the target SPET images. To balance multi-modal inputs, the Mc-Encoder embeds Optimal Multi-modal Transport co-Attention (OMTA) to narrow the heterogeneity gap between image and tabular while capturing their interactions, providing sufficient guidance for reconstruction. In addition, to mitigate semantic distortions, we introduce the Multi-Modal Masked Text Reconstruction (M3TRec), which leverages semantic knowledge extracted from denoised PET images to restore the masked clinical tabular, thereby compelling the network to maintain accurate semantics during reconstruction. To expedite the diffusion process, we further introduce an adversarial diffusive network with a reduced number of diffusion steps. Experiments show that our method achieves the state-of-the-art performance both qualitatively and quantitatively.