# Task-Customized Mixture of Adapters for General Image Fusion

arXiv: https://arxiv.org/abs/2403.12494 · HTML: https://mlsanity.com/paper/2403.12494/

## Key facts

- Cited by 59 papers in the map (data through 14 Oct 2026).
- Placed in Vision mixture-of-experts, one of 12 directions in Mixture-of-experts.
- Builds on 26 of its 54 references that are in the map.
- First posted on arXiv 19 Mar 2024.

## Placed in

- Mixture-of-experts / [Vision mixture-of-experts](https://mlsanity.com/moe/vision-mixture-of-experts/)

## Builds on (most cited first, 26 of 26)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity](https://mlsanity.com/paper/2101.03961/) (2021)
- [Visual Prompt Tuning](https://mlsanity.com/paper/2203.12119/) (2022)
- [GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding](https://mlsanity.com/paper/2006.16668/) (2020)
- [AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognition](https://mlsanity.com/paper/2205.13535/) (2022)
- [Learning multiple visual domains with residual adapters](https://mlsanity.com/paper/1705.08045/) (2017)
- [Target-aware Dual Adversarial Learning and a Multi-scenario Multi-Modality Benchmark to Fuse Infrared and Visible for Object Detection](https://mlsanity.com/paper/2203.16220/) (2022)
- [Scaling & Shifting Your Features: A New Baseline for Efficient Model Tuning](https://mlsanity.com/paper/2210.08823/) (2022)
- [LLVIP: A Visible-infrared Paired Dataset for Low-light Vision](https://mlsanity.com/paper/2108.10831/) (2021)
- [CDDFuse: Correlation-Driven Dual-Branch Feature Decomposition for Multi-Modality Image Fusion](https://mlsanity.com/paper/2211.14461/) (2022)
- [Multimodal Contrastive Learning with LIMoE: the Language-Image Mixture of Experts](https://mlsanity.com/paper/2206.02770/) (2022)
- [DenseFuse: A Fusion Approach to Infrared and Visible Images](https://mlsanity.com/paper/1804.08361/) (2018)
- [Trusted Multi-View Classification with Dynamic Evidential Fusion](https://mlsanity.com/paper/2204.11423/) (2022)
- [DDFM: Denoising Diffusion Model for Multi-Modality Image Fusion](https://mlsanity.com/paper/2303.06840/) (2023)
- [RFN-Nest: An end-to-end residual fusion network for infrared and visible images](https://mlsanity.com/paper/2103.04286/) (2021)
- [Provable Dynamic Fusion for Low-Quality Multimodal Data](https://mlsanity.com/paper/2306.02050/) (2023)
- [Unsupervised Misaligned Infrared and Visible Image Fusion via Cross-Modality Image Generation and Registration](https://mlsanity.com/paper/2205.11876/) (2022)
- [DeepFuse: A Deep Unsupervised Approach for Exposure Fusion with Extreme Exposure Image Pairs](https://mlsanity.com/paper/1712.07384/) (2017)
- [DIDFuse: Deep Image Decomposition for Infrared and Visible Image Fusion](https://mlsanity.com/paper/2003.09210/) (2020)
- [Multi-modal Gated Mixture of Local-to-Global Experts for Dynamic Image Fusion](https://mlsanity.com/paper/2302.01392/) (2023)
- [Benchmarking and Comparing Multi-exposure Image Fusion Algorithms](https://mlsanity.com/paper/2007.15156/) (2020)
- [TransMEF: A Transformer-Based Multi-Exposure Image Fusion Framework using Self-Supervised Multi-Task Learning](https://mlsanity.com/paper/2112.01030/) (2021)
- [SwinFuse: A Residual Swin Transformer Fusion Network for Infrared and Visible Images](https://mlsanity.com/paper/2204.11436/) (2022)

## Built on by (most cited first, 30 of 59)

- [One Model for ALL: Low-Level Task Interaction Is a Key to Task-Agnostic Image Fusion](https://mlsanity.com/paper/2502.19854/) (2025)
- [Task-driven Image Fusion with Learnable Fusion Loss](https://mlsanity.com/paper/2412.03240/) (2024)
- [UltraFusion: Ultra High Dynamic Imaging using Exposure Fusion](https://mlsanity.com/paper/2501.11515/) (2025)
- [Predictive Dynamic Fusion](https://mlsanity.com/paper/2406.04802/) (2024)
- [Efficient Rectified Flow for Image Fusion](https://mlsanity.com/paper/2509.16549/) (2025)
- [Conditional Controllable Image Fusion](https://mlsanity.com/paper/2411.01573/) (2024)
- [IRGPT: Understanding Real-world Infrared Image with Bi-cross-modal Curriculum on Large-scale Benchmark](https://mlsanity.com/paper/2507.14449/) (2025)
- [MAESTRO : Adaptive Sparse Attention and Robust Learning for Multimodal Dynamic Time Series](https://mlsanity.com/paper/2509.25278/) (2025)
- [MMAIF: Multi-task and Multi-degradation All-in-One for Image Fusion with Language Guidance](https://mlsanity.com/paper/2503.14944/) (2025)
- [A Unified Solution to Video Fusion: From Multi-Frame Learning to Benchmarking](https://mlsanity.com/paper/2505.19858/) (2025)
- [VideoFusion: A Spatio-Temporal Collaborative Network for Multi-modal Video Fusion](https://mlsanity.com/paper/2503.23359/) (2025)
- [Shuffle Mamba: State Space Models with Random Shuffle for Multi-Modal Image Fusion](https://mlsanity.com/paper/2409.01728/) (2024)
- [FeRA: Frequency-Energy Constrained Routing for Effective Diffusion Adaptation Fine-Tuning](https://mlsanity.com/paper/2511.17979/) (2025)
- [Test-Time Dynamic Image Fusion](https://mlsanity.com/paper/2411.02840/) (2024)
- [UniFusion: A Unified Image Fusion Framework with Robust Representation and Source-Aware Preservation](https://mlsanity.com/paper/2603.14214/) (2026)
- [Fusion from Decomposition: A Self-Supervised Approach for Image Fusion and Beyond](https://mlsanity.com/paper/2410.12274/) (2024)
- [Retinex-MEF: Retinex-based Glare Effects Aware Unsupervised Multi-Exposure Image Fusion](https://mlsanity.com/paper/2503.07235/) (2025)
- [MMA-UNet: A Multi-Modal Asymmetric UNet Architecture for Infrared and Visible Image Fusion](https://mlsanity.com/paper/2404.17747/) (2024)
- [DM-FNet: Unified multimodal medical image fusion via diffusion process-trained encoder-decoder](https://mlsanity.com/paper/2506.15218/) (2025)
- [Rethinking Efficient Mixture-of-Experts for Remote Sensing Modality-Missing Classification](https://mlsanity.com/paper/2511.11460/) (2025)
- [Self-supervised Multiplex Consensus Mamba for General Image Fusion](https://mlsanity.com/paper/2512.20921/) (2025)
- [Dream-IF: Dynamic Relative EnhAnceMent for Image Fusion](https://mlsanity.com/paper/2503.10109/) (2025)
- [MExD: An Expert-Infused Diffusion Model for Whole-Slide Image Classification](https://mlsanity.com/paper/2503.12401/) (2025)
- [Distilling Textual Priors from LLM to Efficient Image Fusion](https://mlsanity.com/paper/2504.07029/) (2025)
- [CorrMoE: Mixture of Experts with De-stylization Learning for Cross-Scene and Cross-Domain Correspondence Pruning](https://mlsanity.com/paper/2507.11834/) (2025)
- [Text-Guided Channel Perturbation and Pretrained Knowledge Integration for Unified Multi-Modality Image Fusion](https://mlsanity.com/paper/2511.12432/) (2025)
- [MambaVF: State Space Model for Efficient Video Fusion](https://mlsanity.com/paper/2602.06017/) (2026)
- [Lightweight Adaptive Feature Composition for Heterogeneous Downstream Adaptation of Wireless Foundation Models](https://mlsanity.com/paper/2606.10277/) (2026)
- [From 2D Grids to 1D Tokens: Reforming Shared Representations for Multimodal Image Fusion](https://mlsanity.com/paper/2606.12303/) (2026)
- [IVGF: The Fusion-Guided Infrared and Visible General Framework](https://mlsanity.com/paper/2409.00973/) (2024)

## Abstract (arXiv)

General image fusion aims at integrating important information from multi-source images. However, due to the significant cross-task gap, the respective fusion mechanism varies considerably in practice, resulting in limited performance across subtasks. To handle this problem, we propose a novel task-customized mixture of adapters (TC-MoA) for general image fusion, adaptively prompting various fusion tasks in a unified model. We borrow the insight from the mixture of experts (MoE), taking the experts as efficient tuning adapters to prompt a pre-trained foundation model. These adapters are shared across different tasks and constrained by mutual information regularization, ensuring compatibility with different tasks while complementarity for multi-source images. The task-specific routing networks customize these adapters to extract task-specific information from different sources with dynamic dominant intensity, performing adaptive visual feature prompt fusion. Notably, our TC-MoA controls the dominant intensity bias for different fusion tasks, successfully unifying multiple fusion tasks in a single model. Extensive experiments show that TC-MoA outperforms the competing approaches in learning commonalities while retaining compatibility for general image fusion (multi-modal, multi-exposure, and multi-focus), and also demonstrating striking controllability on more generalization experiments. The code is available at https://github.com/YangSun22/TC-MoA .