# Spatial-Spectral Binarized Neural Network for Panchromatic and Multi-spectral Images Fusion

arXiv: https://arxiv.org/abs/2509.23321 · HTML: https://mlsanity.com/paper/2509.23321/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 41 references that are in the map.
- First posted on arXiv 27 Sep 2025.

## Builds on (most cited first, 18 of 18)

- [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013)
- [XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks](https://mlsanity.com/paper/1603.05279/) (2016)
- [Bi-Real Net: Enhancing the Performance of 1-bit CNNs With Improved Representational Capability and Advanced Training Algorithm](https://mlsanity.com/paper/1808.00278/) (2018)
- [Binary Neural Networks: A Survey](https://mlsanity.com/paper/2004.03333/) (2020)
- [Forward and Backward Information Retention for Accurate Binary Neural Networks](https://mlsanity.com/paper/1909.10788/) (2019)
- [What Makes a "Good" Data Augmentation in Knowledge Distillation -- A Statistical Perspective](https://mlsanity.com/paper/2012.02909/) (2020)
- [HyperTransformer: A Textural and Spectral Feature Fusion Transformer for Pansharpening](https://mlsanity.com/paper/2203.02503/) (2022)
- [Basic Binary Convolution Unit for Binarized Image Restoration Network](https://mlsanity.com/paper/2210.00405/) (2022)
- [Content-Adaptive Non-Local Convolution for Remote Sensing Pansharpening](https://mlsanity.com/paper/2404.07543/) (2024)
- [PanFormer: a Transformer Based Model for Pan-sharpening](https://mlsanity.com/paper/2203.02916/) (2022)
- [Binarized Low-light Raw Video Enhancement](https://mlsanity.com/paper/2403.19944/) (2024)
- [Trainability Preserving Neural Pruning](https://mlsanity.com/paper/2207.12534/) (2022)
- [PanFlowNet: A Flow-Based Deep Network for Pan-sharpening](https://mlsanity.com/paper/2305.07774/) (2023)
- [Lipschitz Continuity Retained Binary Neural Network](https://mlsanity.com/paper/2207.06540/) (2022)
- [Frequency-Adaptive Pan-Sharpening with Mixture of Experts](https://mlsanity.com/paper/2401.02151/) (2024)
- [Local-Global Transformer Enhanced Unfolding Network for Pan-sharpening](https://mlsanity.com/paper/2304.14612/) (2023)
- [Pansharpening via Frequency-Aware Fusion Network with Explicit Similarity Constraints](https://mlsanity.com/paper/2207.08602/) (2022)
- [E2FIF: Push the limit of Binarized Deep Imagery Super-resolution using End-to-end Full-precision Information Flow](https://mlsanity.com/paper/2207.06893/) (2022)

## Abstract (arXiv)

Remote sensing pansharpening aims to reconstruct spatial-spectral properties during the fusion of panchromatic (PAN) images and low-resolution multi-spectral (LR-MS) images, finally generating the high-resolution multi-spectral (HR-MS) images. Although deep learning-based models have achieved excellent performance, they often come with high computational complexity, which hinder their applications on resource-limited devices. In this paper, we explore the feasibility of applying the binary neural network (BNN) to pan-sharpening. Nevertheless, there are two main issues with binarizing pan-sharpening models: (i) the binarization will cause serious spectral distortion due to the inconsistent spectral distribution of the PAN/LR-MS images; (ii) the common binary convolution kernel is difficult to adapt to the multi-scale and anisotropic spatial features of remote sensing objects, resulting in serious degradation of contours. To address the above issues, we design the customized spatial-spectral binarized convolution (S2B-Conv), which is composed of the Spectral-Redistribution Mechanism (SRM) and Gabor Spatial Feature Amplifier (GSFA). Specifically, SRM employs an affine transformation, generating its scaling and bias parameters through a dynamic learning process. GSFA, which randomly selects different frequencies and angles within a preset range, enables to better handle multi-scale and-directional spatial features. A series of S2B-Conv form a brand-new binary network for pan-sharpening, dubbed as S2BNet. Extensive quantitative and qualitative experiments have shown our high-efficiency binarized pan-sharpening method can attain a promising performance.