# ReFusion: Learning Image Fusion from Reconstruction with Learnable Loss via Meta-Learning

arXiv: https://arxiv.org/abs/2312.07943 · HTML: https://mlsanity.com/paper/2312.07943/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 90 references that are in the map.
- First posted on arXiv 13 Dec 2023.

## Builds on (most cited first, 27 of 27)

- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [A Survey on Deep Learning in Medical Image Analysis](https://mlsanity.com/paper/1702.05747/) (2017)
- [On First-Order Meta-Learning Algorithms](https://mlsanity.com/paper/1803.02999/) (2018)
- [Meta-Learning in Neural Networks: A Survey](https://mlsanity.com/paper/2004.05439/) (2020)
- [Learning to Reweight Examples for Robust Deep Learning](https://mlsanity.com/paper/1803.09050/) (2018)
- [Meta-SGD: Learning to Learn Quickly for Few-Shot Learning](https://mlsanity.com/paper/1707.09835/) (2017)
- [Target-aware Dual Adversarial Learning and a Multi-scenario Multi-Modality Benchmark to Fuse Infrared and Visible for Object Detection](https://mlsanity.com/paper/2203.16220/) (2022)
- [CDDFuse: Correlation-Driven Dual-Branch Feature Decomposition for Multi-Modality Image Fusion](https://mlsanity.com/paper/2211.14461/) (2022)
- [DenseFuse: A Fusion Approach to Infrared and Visible Images](https://mlsanity.com/paper/1804.08361/) (2018)
- [DDFM: Denoising Diffusion Model for Multi-Modality Image Fusion](https://mlsanity.com/paper/2303.06840/) (2023)
- [Online Meta-Learning](https://mlsanity.com/paper/1902.08438/) (2019)
- [Multi-interactive Feature Learning and a Full-time Multi-modality Benchmark for Image Fusion and Segmentation](https://mlsanity.com/paper/2308.02097/) (2023)
- [RFN-Nest: An end-to-end residual fusion network for infrared and visible images](https://mlsanity.com/paper/2103.04286/) (2021)
- [Evolved Policy Gradients](https://mlsanity.com/paper/1802.04821/) (2018)
- [LRRNet: A Novel Representation Learning Guided Fusion Network for Infrared and Visible Images](https://mlsanity.com/paper/2304.05172/) (2023)
- [Deep Convolutional Neural Network for Multi-modal Image Restoration and Fusion](https://mlsanity.com/paper/1910.04066/) (2019)
- [DIDFuse: Deep Image Decomposition for Infrared and Visible Image Fusion](https://mlsanity.com/paper/2003.09210/) (2020)
- [CoCoNet: Coupled Contrastive Learning Network with Multi-level Feature Ensemble for Multi-modality Image Fusion](https://mlsanity.com/paper/2211.10960/) (2022)
- [Meta-Learning with Task-Adaptive Loss Function for Few-Shot Learning](https://mlsanity.com/paper/2110.03909/) (2021)
- [Deep Gradient Projection Networks for Pan-sharpening](https://mlsanity.com/paper/2103.04584/) (2021)
- [Medical Image Fusion: A survey of the state of the art](https://mlsanity.com/paper/1401.0166/) (2013)
- [Benchmarking and Comparing Multi-exposure Image Fusion Algorithms](https://mlsanity.com/paper/2007.15156/) (2020)
- [HyperTransformer: A Textural and Spectral Feature Fusion Transformer for Pansharpening](https://mlsanity.com/paper/2203.02503/) (2022)
- [MEFLUT: Unsupervised 1D Lookup Tables for Multi-exposure Image Fusion](https://mlsanity.com/paper/2309.11847/) (2023)
- [Interactive Feature Embedding for Infrared and Visible Image Fusion](https://mlsanity.com/paper/2211.04877/) (2022)

## Built on by (most cited first, 15 of 15)

- [Task-driven Image Fusion with Learnable Fusion Loss](https://mlsanity.com/paper/2412.03240/) (2024)
- [Neural Video Compression with Context Modulation](https://mlsanity.com/paper/2505.14541/) (2025)
- [A Unified Solution to Video Fusion: From Multi-Frame Learning to Benchmarking](https://mlsanity.com/paper/2505.19858/) (2025)
- [Shuffle Mamba: State Space Models with Random Shuffle for Multi-Modal Image Fusion](https://mlsanity.com/paper/2409.01728/) (2024)
- [Retinex-MEF: Retinex-based Glare Effects Aware Unsupervised Multi-Exposure Image Fusion](https://mlsanity.com/paper/2503.07235/) (2025)
- [A Comprehensive Review of Techniques, Algorithms, Advancements, Challenges, and Clinical Applications of Multi-modal Medical Image Fusion for Improved Diagnosis](https://mlsanity.com/paper/2505.14715/) (2025)
- [A Semantic-Aware and Multi-Guided Network for Infrared-Visible Image Fusion](https://mlsanity.com/paper/2407.06159/) (2024)
- [Haar Nuclear Norms with Applications to Remote Sensing Imagery Restoration](https://mlsanity.com/paper/2407.08509/) (2024)
- [MambaVF: State Space Model for Efficient Video Fusion](https://mlsanity.com/paper/2602.06017/) (2026)
- [Uncertainty-aware Spatial-Frequency Registration and Fusion for Infrared and Visible Images](https://mlsanity.com/paper/2605.13049/) (2026)
- [RLBayes: a Bayesian Network Structure Learning Algorithm via Reinforcement Learning-Based Search Strategy](https://mlsanity.com/paper/2504.05167/) (2025)
- [Robust Fusion Controller: Degradation-aware Image Fusion with Fine-grained Language Instructions](https://mlsanity.com/paper/2504.05795/) (2025)
- [Rethinking Few-Shot Image Fusion: Granular Ball Priors Enable General-Purpose Deep Fusion](https://mlsanity.com/paper/2504.08937/) (2025)
- [Beyond Spatial-Domain Supervision: A Relation Constrained Space for Multi-Modal Image Fusion](https://mlsanity.com/paper/2609.38968/) (2026)
- [When Integral Meets Decomposition: A Signal-Level Self-Supervised Feature Decompose Paradigm for Multi-Modal Image Fusion](https://mlsanity.com/paper/2609.39004/) (2026)

## Abstract (arXiv)

Image fusion aims to combine information from multiple source images into a single one with more comprehensive informational content. Deep learning-based image fusion algorithms face significant challenges, including the lack of a definitive ground truth and the corresponding distance measurement. Additionally, current manually defined loss functions limit the model's flexibility and generalizability for various fusion tasks. To address these limitations, we propose ReFusion, a unified meta-learning based image fusion framework that dynamically optimizes the fusion loss for various tasks through source image reconstruction. Compared to existing methods, ReFusion employs a parameterized loss function, that allows the training framework to be dynamically adapted according to the specific fusion scenario and task. ReFusion consists of three key components: a fusion module, a source reconstruction module, and a loss proposal module. We employ a meta-learning strategy to train the loss proposal module using the reconstruction loss. This strategy forces the fused image to be more conducive to reconstruct source images, allowing the loss proposal module to generate a adaptive fusion loss that preserves the optimal information from the source images. The update of the fusion module relies on the learnable fusion loss proposed by the loss proposal module. The three modules update alternately, enhancing each other to optimize the fusion loss for different tasks and consistently achieve satisfactory results. Extensive experiments demonstrate that ReFusion is capable of adapting to various tasks, including infrared-visible, medical, multi-focus, and multi-exposure image fusion.