# Text-IF: Leveraging Semantic Text Guidance for Degradation-Aware and Interactive Image Fusion

arXiv: https://arxiv.org/abs/2403.16387 · HTML: https://mlsanity.com/paper/2403.16387/

## Key facts

- Cited by 84 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 43 references that are in the map.
- First posted on arXiv 25 Mar 2024.

## Builds on (most cited first, 18 of 18)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [MUSIQ: Multi-scale Image Quality Transformer](https://mlsanity.com/paper/2108.05997/) (2021)
- [StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery](https://mlsanity.com/paper/2103.17249/) (2021)
- [Exploring CLIP for Assessing the Look and Feel of Images](https://mlsanity.com/paper/2207.12396/) (2022)
- [DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulation](https://mlsanity.com/paper/2110.02711/) (2021)
- [Compositional Visual Generation with Composable Diffusion Models](https://mlsanity.com/paper/2206.01714/) (2022)
- [Target-aware Dual Adversarial Learning and a Multi-scenario Multi-Modality Benchmark to Fuse Infrared and Visible for Object Detection](https://mlsanity.com/paper/2203.16220/) (2022)
- [LLVIP: A Visible-infrared Paired Dataset for Low-light Vision](https://mlsanity.com/paper/2108.10831/) (2021)
- [DF-GAN: A Simple and Effective Baseline for Text-to-Image Synthesis](https://mlsanity.com/paper/2008.05865/) (2020)
- [DDFM: Denoising Diffusion Model for Multi-Modality Image Fusion](https://mlsanity.com/paper/2303.06840/) (2023)
- [Multi-interactive Feature Learning and a Full-time Multi-modality Benchmark for Image Fusion and Segmentation](https://mlsanity.com/paper/2308.02097/) (2023)
- [PromptIR: Prompting for All-in-One Blind Image Restoration](https://mlsanity.com/paper/2306.13090/) (2023)
- [Unsupervised Misaligned Infrared and Visible Image Fusion via Cross-Modality Image Generation and Registration](https://mlsanity.com/paper/2205.11876/) (2022)
- [Masked Image Training for Generalizable Deep Image Denoising](https://mlsanity.com/paper/2303.13132/) (2023)
- [Text to Image Generation with Semantic-Spatial Aware GAN](https://mlsanity.com/paper/2104.00567/) (2021)

## Built on by (most cited first, 30 of 84)

- [Infrared and Visible Image Fusion: From Data Compatibility to Task Adaption](https://mlsanity.com/paper/2501.10761/) (2025)
- [DCEvo: Discriminative Cross-Dimensional Evolutionary Learning for Infrared and Visible Image Fusion](https://mlsanity.com/paper/2503.17673/) (2025)
- [One Model for ALL: Low-Level Task Interaction Is a Key to Task-Agnostic Image Fusion](https://mlsanity.com/paper/2502.19854/) (2025)
- [VLM-E2E: Enhancing End-to-End Autonomous Driving with Multimodal Driver Attention Fusion](https://mlsanity.com/paper/2502.18042/) (2025)
- [Efficient Rectified Flow for Image Fusion](https://mlsanity.com/paper/2509.16549/) (2025)
- [WaveMamba: Wavelet-Driven Mamba Fusion for RGB-Infrared Object Detection](https://mlsanity.com/paper/2507.18173/) (2025)
- [Conditional Controllable Image Fusion](https://mlsanity.com/paper/2411.01573/) (2024)
- [Rethinking Early-Fusion Strategies for Improved Multispectral Object Detection](https://mlsanity.com/paper/2405.16038/) (2024)
- [MMAIF: Multi-task and Multi-degradation All-in-One for Image Fusion with Language Guidance](https://mlsanity.com/paper/2503.14944/) (2025)
- [ControlFusion: A Controllable Image Fusion Framework with Language-Vision Degradation Prompts](https://mlsanity.com/paper/2503.23356/) (2025)
- [LUT-Fuse: Towards Extremely Fast Infrared and Visible Image Fusion via Distillation to Learnable Look-Up Tables](https://mlsanity.com/paper/2509.00346/) (2025)
- [VideoFusion: A Spatio-Temporal Collaborative Network for Multi-modal Video Fusion](https://mlsanity.com/paper/2503.23359/) (2025)
- [PromptHSI: Universal Hyperspectral Image Restoration with Vision-Language Modulated Frequency Adaptation](https://mlsanity.com/paper/2411.15922/) (2024)
- [Deep Unfolding Multi-modal Image Fusion Network via Attribution Analysis](https://mlsanity.com/paper/2502.01467/) (2025)
- [UniFusion: A Unified Image Fusion Framework with Robust Representation and Source-Aware Preservation](https://mlsanity.com/paper/2603.14214/) (2026)
- [Spatial-Frequency Enhanced Mamba for Multi-Modal Image Fusion](https://mlsanity.com/paper/2511.06593/) (2025)
- [Residual Prior-driven Frequency-aware Network for Image Fusion](https://mlsanity.com/paper/2507.06735/) (2025)
- [FlexiD-Fuse: Flexible number of inputs multi-modal medical image fusion based on diffusion model](https://mlsanity.com/paper/2509.09456/) (2025)
- [Towards Unified Semantic and Controllable Image Fusion: A Diffusion Transformer Approach](https://mlsanity.com/paper/2512.07170/) (2025)
- [Bridging Human Evaluation to Infrared and Visible Image Fusion](https://mlsanity.com/paper/2603.03871/) (2026)
- [LINK: Adaptive Modality Interaction for Audio-Visual Video Parsing](https://mlsanity.com/paper/2412.20872/) (2024)
- [Language-Depth Navigated Thermal and Visible Image Fusion](https://mlsanity.com/paper/2503.08676/) (2025)
- [MUG: Pseudo Labeling Augmented Audio-Visual Mamba Network for Audio-Visual Video Parsing](https://mlsanity.com/paper/2507.01384/) (2025)
- [AWM-Fuse: Multi-Modality Image Fusion for Adverse Weather via Global and Local Text Perception](https://mlsanity.com/paper/2508.16881/) (2025)
- [FS-Diff: Semantic guidance and clarity-aware simultaneous multimodal image fusion and super-resolution](https://mlsanity.com/paper/2509.09427/) (2025)
- [Revisiting Generative Infrared and Visible Image Fusion Based on Human Cognitive Laws](https://mlsanity.com/paper/2510.26268/) (2025)
- [Enhancing 3D Semantic Scene Completion with a Refinement Module](https://mlsanity.com/paper/2512.18363/) (2025)
- [EvaNet: Towards More Efficient and Consistent Infrared and Visible Image Fusion Assessment](https://mlsanity.com/paper/2604.02896/) (2026)
- [A Semantic-Aware and Multi-Guided Network for Infrared-Visible Image Fusion](https://mlsanity.com/paper/2407.06159/) (2024)
- [textrmA^textrm2RNet: Adversarial Attack Resilient Network for Robust Infrared and Visible Image Fusion](https://mlsanity.com/paper/2412.09954/) (2024)

## Abstract (arXiv)

Image fusion aims to combine information from different source images to create a comprehensively representative image. Existing fusion methods are typically helpless in dealing with degradations in low-quality source images and non-interactive to multiple subjective and objective needs. To solve them, we introduce a novel approach that leverages semantic text guidance image fusion model for degradation-aware and interactive image fusion task, termed as Text-IF. It innovatively extends the classical image fusion to the text guided image fusion along with the ability to harmoniously address the degradation and interaction issues during fusion. Through the text semantic encoder and semantic interaction fusion decoder, Text-IF is accessible to the all-in-one infrared and visible image degradation-aware processing and the interactive flexible fusion outcomes. In this way, Text-IF achieves not only multi-modal image fusion, but also multi-modal information fusion. Extensive experiments prove that our proposed text guided image fusion strategy has obvious advantages over SOTA methods in the image fusion performance and degradation treatment. The code is available at https://github.com/XunpengYi/Text-IF.