# PAN-Crafter: Learning Modality-Consistent Alignment for PAN-Sharpening

arXiv: https://arxiv.org/abs/2505.23367 · HTML: https://mlsanity.com/paper/2505.23367/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 67 references that are in the map.
- First posted on arXiv 29 May 2025.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [An Introduction to Convolutional Neural Networks](https://mlsanity.com/paper/1511.08458/) (2015)
- [Learning Texture Transformer Network for Image Super-Resolution](https://mlsanity.com/paper/2006.04139/) (2020)
- [Pixel-Adaptive Convolutional Neural Networks](https://mlsanity.com/paper/1904.05373/) (2019)
- [Robust Reference-based Super-Resolution via C2-Matching](https://mlsanity.com/paper/2106.01863/) (2021)
- [Hyperspectral pansharpening: a review](https://mlsanity.com/paper/1504.04531/) (2015)
- [A Multi-Scale and Multi-Depth Convolutional Neural Network for Remote Sensing Imagery Pan-Sharpening](https://mlsanity.com/paper/1712.09809/) (2017)
- [Slide-Transformer: Hierarchical Vision Transformer with Local Self-Attention](https://mlsanity.com/paper/2304.04237/) (2023)
- [FusionMamba: Efficient Remote Sensing Image Fusion with State Space Model](https://mlsanity.com/paper/2404.07932/) (2024)
- [Content-Adaptive Non-Local Convolution for Remote Sensing Pansharpening](https://mlsanity.com/paper/2404.07543/) (2024)
- [SSDiff: Spatial-spectral Integrated Diffusion Model for Remote Sensing Pansharpening](https://mlsanity.com/paper/2404.11537/) (2024)
- [C-DiffSET: Leveraging Latent Diffusion for SAR-to-EO Image Translation with Confidence-Guided Reliable Object Generation](https://mlsanity.com/paper/2411.10788/) (2024)
- [SIPSA-Net: Shift-Invariant Pan Sharpening with Moving Object Alignment for Satellite Imagery](https://mlsanity.com/paper/2105.02400/) (2021)
- [SiamTrans: Zero-Shot Multi-Frame Image Restoration with Pre-Trained Siamese Transformers](https://mlsanity.com/paper/2112.09426/) (2021)
- [Proximal PanNet: A Model-Based Deep Network for Pansharpening](https://mlsanity.com/paper/2203.04286/) (2022)
- [Linearly-evolved Transformer for Pan-sharpening](https://mlsanity.com/paper/2404.12804/) (2024)
- [U-Know-DiffPAN: An Uncertainty-aware Knowledge Distillation Diffusion Framework with Details Enhancement for PAN-Sharpening](https://mlsanity.com/paper/2412.06243/) (2024)
- [Probability-based Global Cross-modal Upsampling for Pansharpening](https://mlsanity.com/paper/2303.13659/) (2023)

## Built on by (most cited first, 3 of 3)

- [G-ZAP: A Generalizable Zero-Shot Framework for Arbitrary-Scale Pansharpening](https://mlsanity.com/paper/2603.14412/) (2026)
- [SatFusion: A Unified Framework for Enhancing Remote Sensing Images via Multi-Frame and Multi-Source Images Fusion](https://mlsanity.com/paper/2510.07905/) (2025)
- [Robust Self-Supervised Cross-Modal Super-Resolution against Real-World Misaligned Observations](https://mlsanity.com/paper/2602.18822/) (2026)

## Abstract (arXiv)

PAN-sharpening aims to fuse high-resolution panchromatic (PAN) images with low-resolution multi-spectral (MS) images to generate high-resolution multi-spectral (HRMS) outputs. However, cross-modality misalignment -- caused by sensor placement, acquisition timing, and resolution disparity -- induces a fundamental challenge. Conventional deep learning methods assume perfect pixel-wise alignment and rely on per-pixel reconstruction losses, leading to spectral distortion, double edges, and blurring when misalignment is present. To address this, we propose PAN-Crafter, a modality-consistent alignment framework that explicitly mitigates the misalignment gap between PAN and MS modalities. At its core, Modality-Adaptive Reconstruction (MARs) enables a single network to jointly reconstruct HRMS and PAN images, leveraging PAN's high-frequency details as auxiliary self-supervision. Additionally, we introduce Cross-Modality Alignment-Aware Attention (CM3A), a novel mechanism that bidirectionally aligns MS texture to PAN structure and vice versa, enabling adaptive feature refinement across modalities. Extensive experiments on multiple benchmark datasets demonstrate that our PAN-Crafter outperforms the most recent state-of-the-art method in all metrics, even with 50.11× faster inference time and 0.63× the memory size. Furthermore, it demonstrates strong generalization performance on unseen satellite datasets, showing its robustness across different conditions.