# Beyond Strict Pairing: Arbitrarily Paired Training for High-Performance Infrared and Visible Image Fusion

arXiv: https://arxiv.org/abs/2603.21820 · HTML: https://mlsanity.com/paper/2603.21820/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 34 of its 56 references that are in the map.
- First posted on arXiv 23 Mar 2026.

## Builds on (most cited first, 30 of 34)

- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [Uformer: A General U-Shaped Transformer for Image Restoration](https://mlsanity.com/paper/2106.03106/) (2021)
- [Target-aware Dual Adversarial Learning and a Multi-scenario Multi-Modality Benchmark to Fuse Infrared and Visible for Object Detection](https://mlsanity.com/paper/2203.16220/) (2022)
- [LLVIP: A Visible-infrared Paired Dataset for Low-light Vision](https://mlsanity.com/paper/2108.10831/) (2021)
- [CDDFuse: Correlation-Driven Dual-Branch Feature Decomposition for Multi-Modality Image Fusion](https://mlsanity.com/paper/2211.14461/) (2022)
- [DenseFuse: A Fusion Approach to Infrared and Visible Images](https://mlsanity.com/paper/1804.08361/) (2018)
- [DDFM: Denoising Diffusion Model for Multi-Modality Image Fusion](https://mlsanity.com/paper/2303.06840/) (2023)
- [GLU-Net: Global-Local Universal Network for Dense Flow and Correspondences](https://mlsanity.com/paper/1912.05524/) (2019)
- [Towards Large-Scale Small Object Detection: Survey and Benchmarks](https://mlsanity.com/paper/2207.14096/) (2022)
- [RFN-Nest: An end-to-end residual fusion network for infrared and visible images](https://mlsanity.com/paper/2103.04286/) (2021)
- [Equivariant Multi-Modality Image Fusion](https://mlsanity.com/paper/2305.11443/) (2023)
- [LRRNet: A Novel Representation Learning Guided Fusion Network for Infrared and Visible Images](https://mlsanity.com/paper/2304.05172/) (2023)
- [PIDNet: A Real-time Semantic Segmentation Network Inspired by PID Controllers](https://mlsanity.com/paper/2206.02066/) (2022)
- [FusionMamba: Dynamic Feature Enhancement for Multimodal Image Fusion with Mamba](https://mlsanity.com/paper/2404.09498/) (2024)
- [Unsupervised Misaligned Infrared and Visible Image Fusion via Cross-Modality Image Generation and Registration](https://mlsanity.com/paper/2205.11876/) (2022)
- [AdaMixer: A Fast-Converging Query-Based Object Detector](https://mlsanity.com/paper/2203.16507/) (2022)
- [Unsupervised Multi-Modal Image Registration via Geometry Preserving Image-to-Image Translation](https://mlsanity.com/paper/2003.08073/) (2020)
- [Unsupervised Learning for Real-World Super-Resolution](https://mlsanity.com/paper/1909.09629/) (2019)
- [Point-Query Quadtree for Crowd Counting, Localization, and More](https://mlsanity.com/paper/2308.13814/) (2023)
- [CrossFuse: A Novel Cross Attention Mechanism based Infrared and Visible Image Fusion Approach](https://mlsanity.com/paper/2406.10581/) (2024)
- [Closing the Loop: Joint Rain Generation and Removal via Disentangled Image Translation](https://mlsanity.com/paper/2103.13660/) (2021)
- [Federated Incremental Semantic Segmentation](https://mlsanity.com/paper/2304.04620/) (2023)
- [Every SAM Drop Counts: Embracing Semantic Priors for Multi-Modality Image Fusion and Beyond](https://mlsanity.com/paper/2503.01210/) (2025)
- [One Model for ALL: Low-Level Task Interaction Is a Key to Task-Agnostic Image Fusion](https://mlsanity.com/paper/2502.19854/) (2025)
- [TGFuse: An Infrared and Visible Image Fusion Approach Based on Transformer and Generative Adversarial Network](https://mlsanity.com/paper/2201.10147/) (2022)
- [Spherical Space Feature Decomposition for Guided Depth Map Super-Resolution](https://mlsanity.com/paper/2303.08942/) (2023)
- [Unpaired Deep Image Dehazing Using Contrastive Disentanglement Learning](https://mlsanity.com/paper/2203.07677/) (2022)
- [Gramformer: Learning Crowd Counting via Graph-Modulated Transformer](https://mlsanity.com/paper/2401.03870/) (2024)
- [Learn from Unpaired Data for Image Restoration: A Variational Bayes Approach](https://mlsanity.com/paper/2204.10090/) (2022)

## Abstract (arXiv)

Infrared and visible image fusion(IVIF) combines complementary modalities while preserving natural textures and salient thermal signatures. Existing solutions predominantly rely on extensive sets of rigidly aligned image pairs for training. However, acquiring such data is often impractical due to the costly and labour-intensive alignment process. Besides, maintaining a rigid pairing setting during training restricts the volume of cross-modal relationships, thereby limiting generalisation performance. To this end, this work challenges the necessity of Strictly Paired Training Paradigm (SPTP) by systematically investigating UnPaired and Arbitrarily Paired Training Paradigms (UPTP and APTP) for high-performance IVIF. We establish a theoretical objective of APTP, reflecting the complementary nature between UPTP and SPTP. More importantly, we develop a practical framework capable of significantly enriching cross-modal relationships even with severely limited and unaligned training data. To validate our propositions, three end-to-end lightweight baselines, alongside a set of innovative loss functions, are designed to cover three classic frameworks (CNN, Transformer, GAN). Comprehensive experiments demonstrate that the proposed APTP and UPTP are feasible and capable of training models on a severely limited and content-inconsistent infrared and visible dataset, achieving performance comparable to that of a dataset 100× larger in SPTP. This finding fundamentally alleviates the cost and difficulty of data collection while enhancing model robustness from the data perspective, delivering a feasible solution for IVIF studies. The code is available at hrefhttps://github.com/yanglinDeng/IVIFunpairtextcolorbluehttps://github.com/yanglinDeng/IVIFunpair.