mlsanityData to 7 Oct 2026

DDFM: Denoising Diffusion Model for Multi-Modality Image Fusion

arXiv 2303.06840 ↗First posted 13 Mar 2023cs.CVcited by 153

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Multi-modality image fusion aims to combine different modalities to produce fused images that retain the complementary features of each modality, such as functional highlights and texture details. To leverage strong generative priors and address challenges such as unstable training and lack of interpretability for GAN-based generative methods, we propose a novel fusion algorithm based on the denoising diffusion probabilistic model (DDPM). The fusion task is formulated as a conditional generation problem under the DDPM sampling framework, which is further divided into an unconditional generation subproblem and a maximum likelihood subproblem. The latter is modeled in a hierarchical Bayesian manner with latent variables and inferred by the expectation-maximization (EM) algorithm. By integrating the inference solution into the diffusion sampling iteration, our method can generate high-quality fused images with natural image generative priors and cross-modality information from source images. Note that all we required is an unconditional pre-trained generative model, and no fine-tuning is needed. Our extensive experiments indicate that our approach yields promising fusion results in infrared-visible image fusion and medical image fusion. The code is available at urlhttps://github.com/Zhaozixiang1228/MMIF-DDFM.

Key facts. Cited by 153 papers in the map (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 35 of its 76 references that are in the map. First posted on arXiv 13 Mar 2023.

Builds on

35 of 76 references.

References · 35

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2020Denoising Diffusion Implicit Modelscited by 9,655
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  5. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  6. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  7. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  8. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  9. 2014Conditional Generative Adversarial Netscited by 2,152
  10. 2020YOLOv4: Optimal Speed and Accuracy of Object Detectioncited by 1,624
  11. 2022Diffusion Posterior Sampling for General Noisy Inverse Problemscited by 1,375
  12. 2016Least Squares Generative Adversarial Networkscited by 1,000
23 more
  1. 2022Denoising Diffusion Restoration Modelscited by 951
  2. 2022Zero-Shot Image Restoration Using Denoising Diffusion Null-Space Modelcited by 574
  3. 2021Tackling the Generative Learning Trilemma with Denoising Diffusion GANscited by 515
  4. 2022Target-aware Dual Adversarial Learning and a Multi-scenario Multi-Modality Benchmark to Fuse Infrared and Visible for Object Detectioncited by 313
  5. 2022CDDFuse: Correlation-Driven Dual-Branch Feature Decomposition for Multi-Modality Image Fusioncited by 222
  6. 2018DenseFuse: A Fusion Approach to Infrared and Visible Imagescited by 180
  7. 2021Dual Attention Suppression Attack: Generate Adversarial Camouflage in Physical Worldcited by 127
  8. 2021RFN-Nest: An end-to-end residual fusion network for infrared and visible imagescited by 109
  9. 2022Unsupervised Misaligned Infrared and Visible Image Fusion via Cross-Modality Image Generation and Registrationcited by 80
  10. 2019Deep Convolutional Neural Network for Multi-modal Image Restoration and Fusioncited by 78
  11. 2020DIDFuse: Deep Image Decomposition for Infrared and Visible Image Fusioncited by 72
  12. 2021Discrete Cosine Transform Network for Guided Depth Map Super-Resolutioncited by 71
  13. 2020NestFuse: An Infrared and Visible Image Fusion Architecture based on Nest Connection and Spatial/Channel Attention Modelscited by 62
  14. 2021Deep Gradient Projection Networks for Pan-sharpeningcited by 44
  15. 2013Medical Image Fusion: A survey of the state of the artcited by 39
  16. 2021Distribution-sensitive Information Retention for Accurate Binary Neural Networkcited by 25
  17. 2021Image Fusion Transformercited by 24
  18. 2023BiBench: Benchmarking and Analyzing Network Binarizationcited by 22
  19. 2022Defensive Patches for Robust Recognition in the Physical Worldcited by 20
  20. 2020Bayesian Fusion for Infrared and Visible Imagescited by 14
  21. 2021Diverse Sample Generation: Pushing the Limit of Generative Data-free Quantizationcited by 11
  22. 2020Deep Convolutional Sparse Coding Networks for Image Fusioncited by 4
  23. 2020FGF-GAN: A Lightweight Generative Adversarial Network for Pansharpening via Fast Guided Filtercited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 21; 2024: 42; 2025: 52; 2026: 38.

153 papers.

Sort:

By area:

  1. 2023Multi-interactive Feature Learning and a Full-time Multi-modality Benchmark for Image Fusion and Segmentationcited by 128
  2. 2023Equivariant Multi-Modality Image Fusioncited by 98
  3. 2024Text-IF: Leveraging Semantic Text Guidance for Degradation-Aware and Interactive Image Fusioncited by 83
  4. 2024Diffusion Models in Low-Level Vision: A Surveycited by 73
  5. 2023Reti-Diff: Illumination Degradation Image Restoration with Retinex-based Latent Diffusion Modelcited by 69
  6. 2024Task-Customized Mixture of Adapters for General Image Fusioncited by 59
  7. 2023A Task-guided, Implicitly-searched and Meta-initialized Deep Model for Image Fusioncited by 47
  8. 2025Infrared and Visible Image Fusion: From Data Compatibility to Task Adaptioncited by 45
  9. 2023Image Restoration Through Generalized Ornstein-Uhlenbeck Bridgecited by 41
  10. 2024Image Fusion via Vision-Language Modelcited by 40
  11. 2024CrossFuse: A Novel Cross Attention Mechanism based Infrared and Visible Image Fusion Approachcited by 38
  12. 2024MambaDFuse: A Mamba-based Dual-phase Model for Multi-modality Image Fusioncited by 35
38 more
  1. 2025Every SAM Drop Counts: Embracing Semantic Priors for Multi-Modality Image Fusion and Beyondcited by 34
  2. 2025DCEvo: Discriminative Cross-Dimensional Evolutionary Learning for Infrared and Visible Image Fusioncited by 34
  3. 2025One Model for ALL: Low-Level Task Interaction Is a Key to Task-Agnostic Image Fusioncited by 32
  4. 2024Text-DiFuse: An Interactive Multi-Modal Image Fusion Framework based on Text-modulated Diffusion Modelcited by 31
  5. 2023Spherical Space Feature Decomposition for Guided Depth Map Super-Resolutioncited by 28
  6. 2024Video Depth without Video Modelscited by 28
  7. 2025ContextFlow: Training-Free Video Object Editing via Adaptive Context Enrichmentcited by 27
  8. 2023Bi-level Dynamic Learning for Jointly Multi-modality Image Fusion and Beyondcited by 26
  9. 2023TextFusion: Unveiling the Power of Textual Semantics for Controllable Image Fusioncited by 26
  10. 2024Task-driven Image Fusion with Learnable Fusion Losscited by 24
  11. 2023FedDiff: Diffusion Model Driven Federated Learning for Multi-Modal and Multi-Clientscited by 23
  12. 2024E2E-MFD: Towards End-to-End Synchronous Multimodal Fusion Detectioncited by 21
  13. 2023From Text to Pixels: A Context-Aware Semantic Synergy Solution for Infrared and Visible Image Fusioncited by 19
  14. 2023TFDet: Target-Aware Fusion for RGB-T Pedestrian Detectioncited by 18
  15. 2023Improving Misaligned Multi-modality Image Fusion with One-stage Progressive Dense Registrationcited by 17
  16. 2024TopoCellGen: Generating Histopathology Cell Topology with a Diffusion Modelcited by 16
  17. 2025Solving Inverse Problems with FLAIRcited by 16
  18. 2023ReFusion: Learning Image Fusion from Reconstruction with Learnable Loss via Meta-Learningcited by 15
  19. 2024MaeFuse: Transferring Omni Features with Pretrained Masked Autoencoders for Infrared and Visible Image Fusion via Guided Trainingcited by 15
  20. 2024A Dynamic Kernel Prior Model for Unsupervised Blind Image Super-Resolutioncited by 15
  21. 2023Scale-Equivariant Imaging: Self-Supervised Learning for Image Super-Resolution and Deblurringcited by 14
  22. 2024All-weather Multi-Modality Image Fusion: Unified Framework and 100k Benchmarkcited by 14
  23. 2025OptiScene: LLM-driven Indoor Scene Layout Generation via Scaled Human-aligned Data Synthesis and Multi-Stage Preference Optimizationcited by 14
  24. 2025Efficient Rectified Flow for Image Fusioncited by 14
  25. 2024Simultaneous Tri-Modal Medical Image Fusion and Super-Resolution using Conditional Diffusion Modelcited by 13
  26. 2024Conditional Controllable Image Fusioncited by 13
  27. 2023Optimal Transport-Guided Conditional Score-Based Diffusion Modelscited by 11
  28. 2024Multi-modal Crowd Counting via a Broker Modalitycited by 11
  29. 2023Language-driven Scene Synthesis using Multi-conditional Diffusion Modelcited by 10
  30. 2024TSJNet: A Multi-modality Target and Semantic Awareness Joint-driven Image Fusion Networkcited by 10
  31. 2024Blind Inversion using Latent Diffusion Priorscited by 10
  32. 2024MMDRFuse: Distilled Mini-Model with Dynamic Refresh for Multi-Modality Image Fusioncited by 10
  33. 2023A Survey of Emerging Applications of Diffusion Probabilistic Models in MRIcited by 9
  34. 2025MMAIF: Multi-task and Multi-degradation All-in-One for Image Fusion with Language Guidancecited by 9
  35. 2025LUT-Fuse: Towards Extremely Fast Infrared and Visible Image Fusion via Distillation to Learnable Look-Up Tablescited by 9
  36. 2023A Spectral Diffusion Prior for Hyperspectral Image Super-Resolutioncited by 8
  37. 2023MDFL: Multi-domain Diffusion-driven Feature Learningcited by 8
  38. 2024Pedestrian Detection in Low-Light Conditions: A Comprehensive Surveycited by 8

50 most cited of 153 All 153 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023DDRF: Denoising Diffusion Model for Remote Sensing Image Fusion
  2. 2023Improving Denoising Diffusion Probabilistic Models via Exploiting Shared Representations
  3. 2025Computationally Efficient Diffusion Models in Medical Imaging: A Comprehensive Review
  4. 2024Diffusion Models For Multi-Modal Generative Modeling
  5. 2022Few-Shot Diffusion Models
  6. 2024A Modular Conditional Diffusion Framework for Image Reconstruction
  7. 2022Unite and Conquer: Plug & Play Multi-Modal Synthesis using Diffusion Models
  8. 2025DSPFusion: Image Fusion via Degradation and Semantic Dual-Prior Guidance