# Dif-Fusion: Towards High Color Fidelity in Infrared and Visible Image Fusion with Diffusion Models

arXiv: https://arxiv.org/abs/2301.08072 · HTML: https://mlsanity.com/paper/2301.08072/

## Key facts

- Cited by 56 papers in the map (data through 14 Oct 2026).
- Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models.
- Builds on 29 of its 97 references that are in the map.
- First posted on arXiv 19 Jan 2023.

## Placed in

- Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/)

## Builds on (most cited first, 29 of 29)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [Image Super-Resolution via Iterative Refinement](https://mlsanity.com/paper/2104.07636/) (2021)
- [RePaint: Inpainting using Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2201.09865/) (2022)
- [Palette: Image-to-Image Diffusion Models](https://mlsanity.com/paper/2111.05826/) (2021)
- [Diffusion Models: A Comprehensive Survey of Methods and Applications](https://mlsanity.com/paper/2209.00796/) (2022)
- [Improved Techniques for Training Score-Based Generative Models](https://mlsanity.com/paper/2006.09011/) (2020)
- [Diffusion Models in Vision: A Survey](https://mlsanity.com/paper/2209.04747/) (2022)
- [Vector Quantized Diffusion Model for Text-to-Image Synthesis](https://mlsanity.com/paper/2111.14822/) (2021)
- [Label-Efficient Semantic Segmentation with Diffusion Models](https://mlsanity.com/paper/2112.03126/) (2021)
- [DiffusionDet: Diffusion Model for Object Detection](https://mlsanity.com/paper/2211.09788/) (2022)
- [Target-aware Dual Adversarial Learning and a Multi-scenario Multi-Modality Benchmark to Fuse Infrared and Visible for Object Detection](https://mlsanity.com/paper/2203.16220/) (2022)
- [Come-Closer-Diffuse-Faster: Accelerating Conditional Diffusion Models for Inverse Problems through Stochastic Contraction](https://mlsanity.com/paper/2112.05146/) (2021)
- [SegDiff: Image Segmentation with Diffusion Probabilistic Models](https://mlsanity.com/paper/2112.00390/) (2021)
- [DenseFuse: A Fusion Approach to Infrared and Visible Images](https://mlsanity.com/paper/1804.08361/) (2018)
- [EGSDE: Unpaired Image-to-Image Translation via Energy-Guided Stochastic Differential Equations](https://mlsanity.com/paper/2207.06635/) (2022)
- [Pyramid Feature Attention Network for Saliency detection](https://mlsanity.com/paper/1903.00179/) (2019)
- [Deep Learning for Hyperspectral Image Classification: An Overview](https://mlsanity.com/paper/1910.12861/) (2019)
- [Cross-modality Person re-identification with Shared-Specific Feature Transfer](https://mlsanity.com/paper/2002.12489/) (2020)
- [NestFuse: An Infrared and Visible Image Fusion Architecture based on Nest Connection and Spatial/Channel Attention Models](https://mlsanity.com/paper/2007.00328/) (2020)
- [Score-based Generative Neural Networks for Large-Scale Optimal Transport](https://mlsanity.com/paper/2110.03237/) (2021)
- [Score-Based Generative Classifiers](https://mlsanity.com/paper/2110.00473/) (2021)
- [Three dimensional Deep Learning approach for remote sensing image classification](https://mlsanity.com/paper/1806.05824/) (2018)
- [SwinFuse: A Residual Swin Transformer Fusion Network for Infrared and Visible Images](https://mlsanity.com/paper/2204.11436/) (2022)
- [The Swiss Army Knife for Image-to-Image Translation: Multi-Task Diffusion Models](https://mlsanity.com/paper/2204.02641/) (2022)

## Built on by (most cited first, 30 of 56)

- [Diffusion Models in Low-Level Vision: A Survey](https://mlsanity.com/paper/2406.11138/) (2024)
- [Infrared and Visible Image Fusion: From Data Compatibility to Task Adaption](https://mlsanity.com/paper/2501.10761/) (2025)
- [Diffusion Policies for Out-of-Distribution Generalization in Offline Reinforcement Learning](https://mlsanity.com/paper/2307.04726/) (2023)
- [MambaDFuse: A Mamba-based Dual-phase Model for Multi-modality Image Fusion](https://mlsanity.com/paper/2404.08406/) (2024)
- [Text-DiFuse: An Interactive Multi-Modal Image Fusion Framework based on Text-modulated Diffusion Model](https://mlsanity.com/paper/2410.23905/) (2024)
- [SpectralDiff: A Generative Framework for Hyperspectral Image Classification with Diffusion Models](https://mlsanity.com/paper/2304.05961/) (2023)
- [Diffusion Models Meet Remote Sensing: Principles, Methods, and Perspectives](https://mlsanity.com/paper/2404.08926/) (2024)
- [FedDiff: Diffusion Model Driven Federated Learning for Multi-Modal and Multi-Clients](https://mlsanity.com/paper/2401.02433/) (2023)
- [Low-light Image Enhancement via CLIP-Fourier Guided Wavelet Diffusion](https://mlsanity.com/paper/2401.03788/) (2024)
- [Efficient Rectified Flow for Image Fusion](https://mlsanity.com/paper/2509.16549/) (2025)
- [Simultaneous Tri-Modal Medical Image Fusion and Super-Resolution using Conditional Diffusion Model](https://mlsanity.com/paper/2404.17357/) (2024)
- [TSJNet: A Multi-modality Target and Semantic Awareness Joint-driven Image Fusion Network](https://mlsanity.com/paper/2402.01212/) (2024)
- [Pedestrian Detection in Low-Light Conditions: A Comprehensive Survey](https://mlsanity.com/paper/2401.07801/) (2024)
- [MagicFace: High-Fidelity Facial Expression Editing with Action-Unit Control](https://mlsanity.com/paper/2501.02260/) (2025)
- [DDRF: Denoising Diffusion Model for Remote Sensing Image Fusion](https://mlsanity.com/paper/2304.04774/) (2023)
- [CrossFuse: Learning Infrared and Visible Image Fusion by Cross-Sensor Top-K Vision Alignment and Beyond](https://mlsanity.com/paper/2502.14493/) (2025)
- [Lightweight Deep Unfolding Networks with Enhanced Robustness for Infrared Small Target Detection](https://mlsanity.com/paper/2509.08205/) (2025)
- [FDiff-Fusion:Denoising diffusion fusion network based on fuzzy learning for 3D medical image segmentation](https://mlsanity.com/paper/2408.02075/) (2024)
- [PanAdapter: Two-Stage Fine-Tuning with Spatial-Spectral Priors Injecting for Pansharpening](https://mlsanity.com/paper/2409.06980/) (2024)
- [Language-Depth Navigated Thermal and Visible Image Fusion](https://mlsanity.com/paper/2503.08676/) (2025)
- [SPAST: Arbitrary Style Transfer with Style Priors via Pre-trained Large-scale Model](https://mlsanity.com/paper/2505.08695/) (2025)
- [FS-Diff: Semantic guidance and clarity-aware simultaneous multimodal image fusion and super-resolution](https://mlsanity.com/paper/2509.09427/) (2025)
- [Revisiting Generative Infrared and Visible Image Fusion Based on Human Cognitive Laws](https://mlsanity.com/paper/2510.26268/) (2025)
- [Graph Representation Learning for Infrared and Visible Image Fusion](https://mlsanity.com/paper/2311.00291/) (2023)
- [MMA-UNet: A Multi-Modal Asymmetric UNet Architecture for Infrared and Visible Image Fusion](https://mlsanity.com/paper/2404.17747/) (2024)
- [CoMoFusion: Fast and High-quality Fusion of Infrared and Visible Image with Consistency Model](https://mlsanity.com/paper/2405.20764/) (2024)
- [HF-VTON: High-Fidelity Virtual Try-On via Consistent Geometric and Semantic Alignment](https://mlsanity.com/paper/2505.19638/) (2025)
- [DM-FNet: Unified multimodal medical image fusion via diffusion process-trained encoder-decoder](https://mlsanity.com/paper/2506.15218/) (2025)
- [FusionCounting: Robust visible-infrared image fusion guided by crowd counting via multi-task learning](https://mlsanity.com/paper/2508.20817/) (2025)
- [FusionRegister: Every Infrared and Visible Image Fusion Deserves Registration](https://mlsanity.com/paper/2603.07667/) (2026)

## Abstract (arXiv)

Color plays an important role in human visual perception, reflecting the spectrum of objects. However, the existing infrared and visible image fusion methods rarely explore how to handle multi-spectral/channel data directly and achieve high color fidelity. This paper addresses the above issue by proposing a novel method with diffusion models, termed as Dif-Fusion, to generate the distribution of the multi-channel input data, which increases the ability of multi-source information aggregation and the fidelity of colors. In specific, instead of converting multi-channel images into single-channel data in existing fusion methods, we create the multi-channel data distribution with a denoising network in a latent space with forward and reverse diffusion process. Then, we use the the denoising network to extract the multi-channel diffusion features with both visible and infrared information. Finally, we feed the multi-channel diffusion features to the multi-channel fusion module to directly generate the three-channel fused image. To retain the texture and intensity information, we propose multi-channel gradient loss and intensity loss. Along with the current evaluation metrics for measuring texture and intensity fidelity, we introduce a new evaluation metric to quantify color fidelity. Extensive experiments indicate that our method is more effective than other state-of-the-art image fusion methods, especially in color fidelity.