# Multi-Head Attention Residual Unfolded Network for Model-Based Pansharpening

arXiv: https://arxiv.org/abs/2409.02675 · HTML: https://mlsanity.com/paper/2409.02675/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 94 references that are in the map.
- First posted on arXiv 4 Sep 2024.

## Builds on (most cited first, 17 of 17)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [Pan-Mamba: Effective pan-sharpening with State Space Model](https://mlsanity.com/paper/2402.12192/) (2024)
- [A Multi-Scale and Multi-Depth Convolutional Neural Network for Remote Sensing Imagery Pan-Sharpening](https://mlsanity.com/paper/1712.09809/) (2017)
- [Deep Gradient Projection Networks for Pan-sharpening](https://mlsanity.com/paper/2103.04584/) (2021)
- [HyperTransformer: A Textural and Spectral Feature Fusion Transformer for Pansharpening](https://mlsanity.com/paper/2203.02503/) (2022)
- [Pansharpening via Detail Injection Based Convolutional Neural Networks](https://mlsanity.com/paper/1806.08898/) (2018)
- [PSGAN: A Generative Adversarial Network for Remote Sensing Image Pan-Sharpening](https://mlsanity.com/paper/1805.03371/) (2018)
- [Content-Adaptive Non-Local Convolution for Remote Sensing Pansharpening](https://mlsanity.com/paper/2404.07543/) (2024)
- [PanFormer: a Transformer Based Model for Pan-sharpening](https://mlsanity.com/paper/2203.02916/) (2022)
- [Local-Global Transformer Enhanced Unfolding Network for Pan-sharpening](https://mlsanity.com/paper/2304.14612/) (2023)
- [A Survey of Pansharpening Methods with A New Band-Decoupled Variational Model](https://mlsanity.com/paper/1606.05703/) (2016)
- [Implicit Neural Feature Fusion Function for Multispectral and Hyperspectral Image Fusion](https://mlsanity.com/paper/2307.07288/) (2023)

## Built on by (most cited first, 2 of 2)

- [Nonlocal Retinex-Based Variational Model and its Deep Unfolding Twin for Low-Light Image Enhancement](https://mlsanity.com/paper/2504.07810/) (2025)
- [GSPan: A Continuous Gaussian Primitive Representation for Arbitrary-Scale Pansharpening](https://mlsanity.com/paper/2606.17722/) (2026)

## Abstract (arXiv)

The objective of pansharpening and hypersharpening is to accurately combine a high-resolution panchromatic (PAN) image with a low-resolution multispectral (MS) or hyperspectral (HS) image, respectively. Unfolding fusion methods integrate the powerful representation capabilities of deep learning with the robustness of model-based approaches. These techniques involve unrolling the steps of the optimization scheme derived from the minimization of an energy into a deep learning framework, resulting in efficient and highly interpretable architectures. In this paper, we propose a model-based deep unfolded method for satellite image fusion. Our approach is based on a variational formulation that incorporates the classic observation model for MS/HS data, a high-frequency injection constraint based on the PAN image, and an arbitrary convex prior. For the unfolding stage, we introduce upsampling and downsampling layers that use geometric information encoded in the PAN image through residual networks. The backbone of our method is a multi-head attention residual network (MARNet), which replaces the proximity operator in the optimization scheme and combines multiple head attentions with residual learning to exploit image self-similarities via nonlocal operators defined in terms of patches. Additionally, we incorporate a post-processing module based on the MARNet architecture to further enhance the quality of the fused images. Experimental results on PRISMA, Quickbird, and WorldView2 datasets demonstrate the superior performance of our method and its ability to generalize across different sensor configurations and varying spatial and spectral resolutions. The source code will be available at https://github.com/TAMI-UIB/MARNet.