# DIFF-MF: A Difference-Driven Channel-Spatial State Space Model for Multi-Modal Image Fusion

arXiv: https://arxiv.org/abs/2601.05538 · HTML: https://mlsanity.com/paper/2601.05538/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 30 of its 57 references that are in the map.
- First posted on arXiv 9 Jan 2026.

## Builds on (most cited first, 30 of 30)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality](https://mlsanity.com/paper/2405.21060/) (2024)
- [Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model](https://mlsanity.com/paper/2401.09417/) (2024)
- [VMamba: Visual State Space Model](https://mlsanity.com/paper/2401.10166/) (2024)
- [A Survey on Visual Transformer](https://mlsanity.com/paper/2012.12556/) (2020)
- [Combining Recurrent, Convolutional, and Continuous-time Models with Linear State-Space Layers](https://mlsanity.com/paper/2110.13985/) (2021)
- [Target-aware Dual Adversarial Learning and a Multi-scenario Multi-Modality Benchmark to Fuse Infrared and Visible for Object Detection](https://mlsanity.com/paper/2203.16220/) (2022)
- [SegMamba: Long-range Sequential Modeling Mamba For 3D Medical Image Segmentation](https://mlsanity.com/paper/2401.13560/) (2024)
- [CDDFuse: Correlation-Driven Dual-Branch Feature Decomposition for Multi-Modality Image Fusion](https://mlsanity.com/paper/2211.14461/) (2022)
- [DenseFuse: A Fusion Approach to Infrared and Visible Images](https://mlsanity.com/paper/1804.08361/) (2018)
- [DDFM: Denoising Diffusion Model for Multi-Modality Image Fusion](https://mlsanity.com/paper/2303.06840/) (2023)
- [Drone-based RGB-Infrared Cross-Modality Vehicle Detection via Uncertainty-Aware Learning](https://mlsanity.com/paper/2003.02437/) (2020)
- [Multi-interactive Feature Learning and a Full-time Multi-modality Benchmark for Image Fusion and Segmentation](https://mlsanity.com/paper/2308.02097/) (2023)
- [ChangeMamba: Remote Sensing Change Detection With Spatiotemporal State Space Model](https://mlsanity.com/paper/2404.03425/) (2024)
- [RFN-Nest: An end-to-end residual fusion network for infrared and visible images](https://mlsanity.com/paper/2103.04286/) (2021)
- [Equivariant Multi-Modality Image Fusion](https://mlsanity.com/paper/2305.11443/) (2023)
- [FusionMamba: Dynamic Feature Enhancement for Multimodal Image Fusion with Mamba](https://mlsanity.com/paper/2404.09498/) (2024)
- [DeepFuse: A Deep Unsupervised Approach for Exposure Fusion with Extreme Exposure Image Pairs](https://mlsanity.com/paper/1712.07384/) (2017)
- [DIDFuse: Deep Image Decomposition for Infrared and Visible Image Fusion](https://mlsanity.com/paper/2003.09210/) (2020)
- [NestFuse: An Infrared and Visible Image Fusion Architecture based on Nest Connection and Spatial/Channel Attention Models](https://mlsanity.com/paper/2007.00328/) (2020)
- [MambaDFuse: A Mamba-based Dual-phase Model for Multi-modality Image Fusion](https://mlsanity.com/paper/2404.08406/) (2024)
- [TransMEF: A Transformer-Based Multi-Exposure Image Fusion Framework using Self-Supervised Multi-Task Learning](https://mlsanity.com/paper/2112.01030/) (2021)
- [Image Fusion Transformer](https://mlsanity.com/paper/2107.09011/) (2021)
- [MTMamba: Enhancing Multi-Task Dense Scene Understanding by Mamba-Based Decoders](https://mlsanity.com/paper/2407.02228/) (2024)
- [PPT Fusion: Pyramid Patch Transformerfor a Case Study in Image Fusion](https://mlsanity.com/paper/2107.13967/) (2021)
- [Interactive Feature Embedding for Infrared and Visible Image Fusion](https://mlsanity.com/paper/2211.04877/) (2022)
- [Graph Representation Learning for Infrared and Visible Image Fusion](https://mlsanity.com/paper/2311.00291/) (2023)

## Abstract (arXiv)

Multi-modal image fusion aims to integrate complementary information from multiple source images to produce high-quality fused images with enriched content. Although existing approaches based on state space model have achieved satisfied performance with high computational efficiency, they tend to either over-prioritize infrared intensity at the cost of visible details, or conversely, preserve visible structure while diminishing thermal target salience. To overcome these challenges, we propose DIFF-MF, a novel difference-driven channel-spatial state space model for multi-modal image fusion. Our approach leverages feature discrepancy maps between modalities to guide feature extraction, followed by a fusion process across both channel and spatial dimensions. In the channel dimension, a channel-exchange module enhances channel-wise interaction through cross-attention dual state space modeling, enabling adaptive feature reweighting. In the spatial dimension, a spatial-exchange module employs cross-modal state space scanning to achieve comprehensive spatial fusion. By efficiently capturing global dependencies while maintaining linear computational complexity, DIFF-MF effectively integrates complementary multi-modal features. Experimental results on the driving scenarios and low-altitude UAV datasets demonstrate that our method outperforms existing approaches in both visual quality and quantitative evaluation.