# CrossDiff: Exploring Self-Supervised Representation of Pansharpening via Cross-Predictive Diffusion Model

arXiv: https://arxiv.org/abs/2401.05153 · HTML: https://mlsanity.com/paper/2401.05153/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Inverse problems and image restoration, one of 14 directions in Diffusion models.
- Builds on 10 of its 56 references that are in the map.
- First posted on arXiv 10 Jan 2024.

## Placed in

- Diffusion models / [Inverse problems and image restoration](https://mlsanity.com/diffusion/inverse-problems-and-image-restoration/)

## Builds on (most cited first, 10 of 10)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [Label-Efficient Semantic Segmentation with Diffusion Models](https://mlsanity.com/paper/2112.03126/) (2021)
- [Adversarial score matching and improved sampling for image generation](https://mlsanity.com/paper/2009.05475/) (2020)
- [DDPM-CD: Denoising Diffusion Probabilistic Models as Feature Extractors for Change Detection](https://mlsanity.com/paper/2206.11892/) (2022)
- [Energy-Inspired Self-Supervised Pretraining for Vision Models](https://mlsanity.com/paper/2302.01384/) (2023)

## Built on by (most cited first, 6 of 6)

- [When Images Look Right and Retrieve Wrong: Coverage-Guided Cross-Scale Re-Indexing for Knowledge-Faithful Generative Perception](https://mlsanity.com/paper/2608.20810/) (2026)
- [Diffusion Models for Hyperspectral Image Analysis: A Comprehensive Review](https://mlsanity.com/paper/2505.11158/) (2025)
- [Enpowering Your Pansharpening Models with Generalizability: Unified Distribution is All You Need](https://mlsanity.com/paper/2510.22217/) (2025)
- [Two-Stage Random Alternation Framework for One-Shot Pansharpening](https://mlsanity.com/paper/2505.06576/) (2025)
- [Progressive Alignment Degradation Learning for Pansharpening](https://mlsanity.com/paper/2506.20179/) (2025)
- [CC-Pan: Channel-wise Compression based Diffusion for Efficient Pan-Sharpening](https://mlsanity.com/paper/2602.04473/) (2026)

## Abstract (arXiv)

Fusion of a panchromatic (PAN) image and corresponding multispectral (MS) image is also known as pansharpening, which aims to combine abundant spatial details of PAN and spectral information of MS. Due to the absence of high-resolution MS images, available deep-learning-based methods usually follow the paradigm of training at reduced resolution and testing at both reduced and full resolution. When taking original MS and PAN images as inputs, they always obtain sub-optimal results due to the scale variation. In this paper, we propose to explore the self-supervised representation of pansharpening by designing a cross-predictive diffusion model, named CrossDiff. It has two-stage training. In the first stage, we introduce a cross-predictive pretext task to pre-train the UNet structure based on conditional DDPM, while in the second stage, the encoders of the UNets are frozen to directly extract spatial and spectral features from PAN and MS, and only the fusion head is trained to adapt for pansharpening task. Extensive experiments show the effectiveness and superiority of the proposed model compared with state-of-the-art supervised and unsupervised methods. Besides, the cross-sensor experiments also verify the generalization ability of proposed self-supervised representation learners for other satellite's datasets. We will release our code for reproducibility.