# MESSFN : a Multi-level and Enhanced Spectral-Spatial Fusion Network for Pan-sharpening

arXiv: https://arxiv.org/abs/2109.09937 · HTML: https://mlsanity.com/paper/2109.09937/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 62 references that are in the map.
- First posted on arXiv 21 Sep 2021.

## Builds on (most cited first, 13 of 13)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Image Super-Resolution Using Very Deep Residual Channel Attention Networks](https://mlsanity.com/paper/1807.02758/) (2018)
- [Image Super-Resolution Using Deep Convolutional Networks](https://mlsanity.com/paper/1501.00092/) (2014)
- [ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks](https://mlsanity.com/paper/1910.03151/) (2019)
- [Early Convolutions Help Transformers See Better](https://mlsanity.com/paper/2106.14881/) (2021)
- [Channel-wise and Spatial Feature Modulation Network for Single Image Super-Resolution](https://mlsanity.com/paper/1809.11130/) (2018)
- [Remote Sensing Image Fusion Based on Two-stream Fusion Network](https://mlsanity.com/paper/1711.02549/) (2017)
- [PSGAN: A Generative Adversarial Network for Remote Sensing Image Pan-Sharpening](https://mlsanity.com/paper/1805.03371/) (2018)
- [Rethinking CNN-Based Pansharpening: Guided Colorization of Panchromatic Images via GANs](https://mlsanity.com/paper/2006.16644/) (2020)

## Abstract (arXiv)

Dominant pan-sharpening frameworks simply concatenate the MS stream and the PAN stream once at a specific level. This way of fusion neglects the multi-level spectral-spatial correlation between the two streams, which is vital to improving the fusion performance. In consideration of this, we propose a Multi-level and Enhanced Spectral-Spatial Fusion Network (MESSFN) with the following innovations: First, to fully exploit and strengthen the above correlation, a Hierarchical Multi-level Fusion Architecture (HMFA) is carefully designed. A novel Spectral-Spatial (SS) stream is established to hierarchically derive and fuse the multi-level prior spectral and spatial expertise from the MS stream and the PAN stream. This helps the SS stream master a joint spectral-spatial representation in the hierarchical network for better modeling the fusion relationship. Second, to provide superior expertise, consequently, based on the intrinsic characteristics of the MS image and the PAN image, two feature extraction blocks are specially developed. In the MS stream, a Residual Spectral Attention Block (RSAB) is proposed to mine the potential spectral correlations between different spectra of the MS image through adjacent cross-spectrum interaction. While in the PAN stream, a Residual Multi-scale Spatial Attention Block (RMSAB) is proposed to capture multi-scale information and reconstruct precise high-frequency details from the PAN image through an improved spatial attention-based inception structure. The spectral and spatial feature representations are enhanced. Extensive experiments on two datasets demonstrate that the proposed network is competitive with or better than state-of-the-art methods. Our code can be found in github.