# TGFuse: An Infrared and Visible Image Fusion Approach Based on Transformer and Generative Adversarial Network

arXiv: https://arxiv.org/abs/2201.10147 · HTML: https://mlsanity.com/paper/2201.10147/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 57 references that are in the map.
- First posted on arXiv 25 Jan 2022.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Least Squares Generative Adversarial Networks](https://mlsanity.com/paper/1611.04076/) (2016)
- [Pre-Trained Image Processing Transformer](https://mlsanity.com/paper/2012.00364/) (2020)
- [DenseFuse: A Fusion Approach to Infrared and Visible Images](https://mlsanity.com/paper/1804.08361/) (2018)
- [BEGAN: Boundary Equilibrium Generative Adversarial Networks](https://mlsanity.com/paper/1703.10717/) (2017)
- [RFN-Nest: An end-to-end residual fusion network for infrared and visible images](https://mlsanity.com/paper/2103.04286/) (2021)
- [PD-GAN: Probabilistic Diverse GAN for Image Inpainting](https://mlsanity.com/paper/2105.02201/) (2021)
- [DeepFuse: A Deep Unsupervised Approach for Exposure Fusion with Extreme Exposure Image Pairs](https://mlsanity.com/paper/1712.07384/) (2017)
- [NestFuse: An Infrared and Visible Image Fusion Architecture based on Nest Connection and Spatial/Channel Attention Models](https://mlsanity.com/paper/2007.00328/) (2020)
- [High-Resolution Photorealistic Image Translation in Real-Time: A Laplacian Pyramid Translation Network](https://mlsanity.com/paper/2105.09188/) (2021)
- [MDLatLRR: A novel decomposition method for infrared and visible image fusion](https://mlsanity.com/paper/1811.02291/) (2018)
- [Infrared and Visible Image Fusion using a Deep Learning Framework](https://mlsanity.com/paper/1804.06992/) (2018)
- [Infrared and Visible Image Fusion with ResNet and zero-phase component analysis](https://mlsanity.com/paper/1806.07119/) (2018)
- [An Accelerated Correlation Filter Tracker](https://mlsanity.com/paper/1912.02854/) (2019)

## Built on by (most cited first, 29 of 29)

- [Infrared and Visible Image Fusion: From Data Compatibility to Task Adaption](https://mlsanity.com/paper/2501.10761/) (2025)
- [Generative-based Fusion Mechanism for Multi-Modal Tracking](https://mlsanity.com/paper/2309.01728/) (2023)
- [Text-DiFuse: An Interactive Multi-Modal Image Fusion Framework based on Text-modulated Diffusion Model](https://mlsanity.com/paper/2410.23905/) (2024)
- [TextFusion: Unveiling the Power of Textual Semantics for Controllable Image Fusion](https://mlsanity.com/paper/2312.14209/) (2023)
- [Simultaneous Tri-Modal Medical Image Fusion and Super-Resolution using Conditional Diffusion Model](https://mlsanity.com/paper/2404.17357/) (2024)
- [FusionBooster: A Unified Image Fusion Boosting Paradigm](https://mlsanity.com/paper/2305.05970/) (2023)
- [TSJNet: A Multi-modality Target and Semantic Awareness Joint-driven Image Fusion Network](https://mlsanity.com/paper/2402.01212/) (2024)
- [UMCFuse: A Unified Multiple Complex Scenes Infrared and Visible Image Fusion Framework](https://mlsanity.com/paper/2402.02096/) (2024)
- [Spatial-Frequency Enhanced Mamba for Multi-Modal Image Fusion](https://mlsanity.com/paper/2511.06593/) (2025)
- [Language-Depth Navigated Thermal and Visible Image Fusion](https://mlsanity.com/paper/2503.08676/) (2025)
- [AWM-Fuse: Multi-Modality Image Fusion for Adverse Weather via Global and Local Text Perception](https://mlsanity.com/paper/2508.16881/) (2025)
- [FS-Diff: Semantic guidance and clarity-aware simultaneous multimodal image fusion and super-resolution](https://mlsanity.com/paper/2509.09427/) (2025)
- [GrFormer: A Novel Transformer on Grassmann Manifold for Infrared and Visible Image Fusion](https://mlsanity.com/paper/2506.14384/) (2025)
- [FusionRegister: Every Infrared and Visible Image Fusion Deserves Registration](https://mlsanity.com/paper/2603.07667/) (2026)
- [EvaNet: Towards More Efficient and Consistent Infrared and Visible Image Fusion Assessment](https://mlsanity.com/paper/2604.02896/) (2026)
- [DePF: A Novel Fusion Approach based on Decomposition Pooling for Infrared and Visible Images](https://mlsanity.com/paper/2305.17376/) (2023)
- [EViT: An Eagle Vision Transformer with Bi-Fovea Self-Attention](https://mlsanity.com/paper/2310.06629/) (2023)
- [MATCNN: Infrared and Visible Image Fusion Method Based on Multi-scale CNN with Attention Transformer](https://mlsanity.com/paper/2502.01959/) (2025)
- [MISFIT-V: Misaligned Image Synthesis and Fusion using Information from Thermal and Visual](https://mlsanity.com/paper/2309.13216/) (2023)
- [Rethinking the Evaluation of Visible and Infrared Image Fusion](https://mlsanity.com/paper/2410.06811/) (2024)
- [Infrared-Assisted Single-Stage Framework for Joint Restoration and Fusion of Visible and Infrared Images under Hazy Conditions](https://mlsanity.com/paper/2411.12586/) (2024)
- [MultiTaskVIF: Segmentation-oriented visible and infrared image fusion via multi-task learning](https://mlsanity.com/paper/2505.06665/) (2025)
- [Multimodal Medical Image Classification via Synergistic Learning Pre-training](https://mlsanity.com/paper/2509.17492/) (2025)
- [SSVIF: Self-Supervised Segmentation-Oriented Visible and Infrared Image Fusion](https://mlsanity.com/paper/2509.22450/) (2025)
- [Missing No More: Dictionary-Guided Cross-Modal Image Fusion under Missing Infrared](https://mlsanity.com/paper/2603.08018/) (2026)
- [TriFusion-SR: Joint Tri-Modal Medical Image Fusion and SR](https://mlsanity.com/paper/2603.09702/) (2026)
- [Beyond Strict Pairing: Arbitrarily Paired Training for High-Performance Infrared and Visible Image Fusion](https://mlsanity.com/paper/2603.21820/) (2026)
- [Heterogeneous 2D/1D Signal Representation Fusion for Underwater Acoustic Modulation Recognition Under Distribution Shift](https://mlsanity.com/paper/2606.23702/) (2026)
- [P2Fusion: Prompt-based Progressive Infrared-Visible Image Fusion via Dual-Prior Distillation](https://mlsanity.com/paper/2608.13045/) (2026)

## Abstract (arXiv)

The end-to-end image fusion framework has achieved promising performance, with dedicated convolutional networks aggregating the multi-modal local appearance. However, long-range dependencies are directly neglected in existing CNN fusion approaches, impeding balancing the entire image-level perception for complex scenario fusion. In this paper, therefore, we propose an infrared and visible image fusion algorithm based on a lightweight transformer module and adversarial learning. Inspired by the global interaction power, we use the transformer technique to learn the effective global fusion relations. In particular, shallow features extracted by CNN are interacted in the proposed transformer fusion module to refine the fusion relationship within the spatial scope and across channels simultaneously. Besides, adversarial learning is designed in the training process to improve the output discrimination via imposing competitive consistency from the inputs, reflecting the specific characteristics in infrared and visible images. The experimental performance demonstrates the effectiveness of the proposed modules, with superior improvement against the state-of-the-art, generalising a novel paradigm via transformer and adversarial learning in the fusion task.