# LRRNet: A Novel Representation Learning Guided Fusion Network for Infrared and Visible Images

arXiv: https://arxiv.org/abs/2304.05172 · HTML: https://mlsanity.com/paper/2304.05172/

## Key facts

- Cited by 87 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 55 references that are in the map.
- First posted on arXiv 11 Apr 2023.

## Builds on (most cited first, 18 of 18)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Image Segmentation Using Deep Learning: A Survey](https://mlsanity.com/paper/2001.05566/) (2020)
- [DenseFuse: A Fusion Approach to Infrared and Visible Images](https://mlsanity.com/paper/1804.08361/) (2018)
- [RGB-T Object Tracking:Benchmark and Baseline](https://mlsanity.com/paper/1805.08982/) (2018)
- [RFN-Nest: An end-to-end residual fusion network for infrared and visible images](https://mlsanity.com/paper/2103.04286/) (2021)
- [LasHeR: A Large-scale High-diversity Benchmark for RGBT Tracking](https://mlsanity.com/paper/2104.13202/) (2021)
- [Deep Convolutional Neural Network for Multi-modal Image Restoration and Fusion](https://mlsanity.com/paper/1910.04066/) (2019)
- [RGB-T Image Saliency Detection via Collaborative Graph Learning](https://mlsanity.com/paper/1905.06741/) (2019)
- [MDLatLRR: A novel decomposition method for infrared and visible image fusion](https://mlsanity.com/paper/1811.02291/) (2018)
- [Challenge-Aware RGBT Tracking](https://mlsanity.com/paper/2007.13143/) (2020)
- [Infrared and Visible Image Fusion using a Deep Learning Framework](https://mlsanity.com/paper/1804.06992/) (2018)
- [Learned Convolutional Sparse Coding](https://mlsanity.com/paper/1711.00328/) (2017)
- [Exploring Fusion Strategies for Accurate RGBT Visual Object Tracking](https://mlsanity.com/paper/2201.08673/) (2022)
- [Convolutional Dictionary Learning via Local Processing](https://mlsanity.com/paper/1705.03239/) (2017)
- [Infrared and Visible Image Fusion with ResNet and zero-phase component analysis](https://mlsanity.com/paper/1806.07119/) (2018)
- [Visual Object Tracking on Multi-modal RGB-D Videos: A Review](https://mlsanity.com/paper/2201.09207/) (2022)
- [AFAT: Adaptive Failure-Aware Tracker for Robust Visual Object Tracking](https://mlsanity.com/paper/2005.13708/) (2020)

## Built on by (most cited first, 30 of 87)

- [Multi-interactive Feature Learning and a Full-time Multi-modality Benchmark for Image Fusion and Segmentation](https://mlsanity.com/paper/2308.02097/) (2023)
- [Equivariant Multi-Modality Image Fusion](https://mlsanity.com/paper/2305.11443/) (2023)
- [Fusion-Mamba for Cross-modality Object Detection](https://mlsanity.com/paper/2404.09146/) (2024)
- [A Task-guided, Implicitly-searched and Meta-initialized Deep Model for Image Fusion](https://mlsanity.com/paper/2305.15862/) (2023)
- [Infrared and Visible Image Fusion: From Data Compatibility to Task Adaption](https://mlsanity.com/paper/2501.10761/) (2025)
- [Image Fusion via Vision-Language Model](https://mlsanity.com/paper/2402.02235/) (2024)
- [CrossFuse: A Novel Cross Attention Mechanism based Infrared and Visible Image Fusion Approach](https://mlsanity.com/paper/2406.10581/) (2024)
- [Every SAM Drop Counts: Embracing Semantic Priors for Multi-Modality Image Fusion and Beyond](https://mlsanity.com/paper/2503.01210/) (2025)
- [DCEvo: Discriminative Cross-Dimensional Evolutionary Learning for Infrared and Visible Image Fusion](https://mlsanity.com/paper/2503.17673/) (2025)
- [One Model for ALL: Low-Level Task Interaction Is a Key to Task-Agnostic Image Fusion](https://mlsanity.com/paper/2502.19854/) (2025)
- [Text-DiFuse: An Interactive Multi-Modal Image Fusion Framework based on Text-modulated Diffusion Model](https://mlsanity.com/paper/2410.23905/) (2024)
- [TextFusion: Unveiling the Power of Textual Semantics for Controllable Image Fusion](https://mlsanity.com/paper/2312.14209/) (2023)
- [E2E-MFD: Towards End-to-End Synchronous Multimodal Fusion Detection](https://mlsanity.com/paper/2403.09323/) (2024)
- [Improving Misaligned Multi-modality Image Fusion with One-stage Progressive Dense Registration](https://mlsanity.com/paper/2308.11165/) (2023)
- [ReFusion: Learning Image Fusion from Reconstruction with Learnable Loss via Meta-Learning](https://mlsanity.com/paper/2312.07943/) (2023)
- [Efficient Rectified Flow for Image Fusion](https://mlsanity.com/paper/2509.16549/) (2025)
- [All-weather Multi-Modality Image Fusion: Unified Framework and 100k Benchmark](https://mlsanity.com/paper/2402.02090/) (2024)
- [Improving Expressivity of GNNs with Subgraph-specific Factor Embedded Normalization](https://mlsanity.com/paper/2305.19903/) (2023)
- [FusionBooster: A Unified Image Fusion Boosting Paradigm](https://mlsanity.com/paper/2305.05970/) (2023)
- [Bridging the Gap between Multi-focus and Multi-modal: A Focused Integration Framework for Multi-modal Image Fusion](https://mlsanity.com/paper/2311.01886/) (2023)
- [TSJNet: A Multi-modality Target and Semantic Awareness Joint-driven Image Fusion Network](https://mlsanity.com/paper/2402.01212/) (2024)
- [Infrared and Visible Image Fusion with Language-Driven Loss in CLIP Embedding Space](https://mlsanity.com/paper/2402.16267/) (2024)
- [UMCFuse: A Unified Multiple Complex Scenes Infrared and Visible Image Fusion Framework](https://mlsanity.com/paper/2402.02096/) (2024)
- [Instruction-Driven Fusion of Infrared-Visible Images: Tailoring for Diverse Downstream Tasks](https://mlsanity.com/paper/2411.09387/) (2024)
- [ControlFusion: A Controllable Image Fusion Framework with Language-Vision Degradation Prompts](https://mlsanity.com/paper/2503.23356/) (2025)
- [A Unified Solution to Video Fusion: From Multi-Frame Learning to Benchmarking](https://mlsanity.com/paper/2505.19858/) (2025)
- [LUT-Fuse: Towards Extremely Fast Infrared and Visible Image Fusion via Distillation to Learnable Look-Up Tables](https://mlsanity.com/paper/2509.00346/) (2025)
- [VideoFusion: A Spatio-Temporal Collaborative Network for Multi-modal Video Fusion](https://mlsanity.com/paper/2503.23359/) (2025)
- [Deep Unfolding Multi-modal Image Fusion Network via Attribution Analysis](https://mlsanity.com/paper/2502.01467/) (2025)
- [DEAL: Data-Efficient Adversarial Learning for High-Quality Infrared Imaging](https://mlsanity.com/paper/2503.00905/) (2025)

## Abstract (arXiv)

Deep learning based fusion methods have been achieving promising performance in image fusion tasks. This is attributed to the network architecture that plays a very important role in the fusion process. However, in general, it is hard to specify a good fusion architecture, and consequently, the design of fusion networks is still a black art, rather than science. To address this problem, we formulate the fusion task mathematically, and establish a connection between its optimal solution and the network architecture that can implement it. This approach leads to a novel method proposed in the paper of constructing a lightweight fusion network. It avoids the time-consuming empirical network design by a trial-and-test strategy. In particular we adopt a learnable representation approach to the fusion task, in which the construction of the fusion network architecture is guided by the optimisation algorithm producing the learnable model. The low-rank representation (LRR) objective is the foundation of our learnable model. The matrix multiplications, which are at the heart of the solution are transformed into convolutional operations, and the iterative process of optimisation is replaced by a special feed-forward network. Based on this novel network architecture, an end-to-end lightweight fusion network is constructed to fuse infrared and visible light images. Its successful training is facilitated by a detail-to-semantic information loss function proposed to preserve the image details and to enhance the salient features of the source images. Our experiments show that the proposed fusion network exhibits better fusion performance than the state-of-the-art fusion methods on public datasets. Interestingly, our network requires a fewer training parameters than other existing methods. The codes are available at https://github.com/hli1221/imagefusion-LRRNet