# Hyperspectral and multispectral image fusion with arbitrary resolution through self-supervised representations

arXiv: https://arxiv.org/abs/2405.17818 · HTML: https://mlsanity.com/paper/2405.17818/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 70 references that are in the map.
- First posted on arXiv 28 May 2024.

## Builds on (most cited first, 18 of 18)

- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Learning Continuous Image Representation with Local Implicit Image Function](https://mlsanity.com/paper/2012.09161/) (2020)
- [Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapes](https://mlsanity.com/paper/2101.10994/) (2021)
- [Beyond Periodicity: Towards a Unifying Framework for Activations in Coordinate-MLPs](https://mlsanity.com/paper/2111.15135/) (2021)
- [Image Generators with Conditionally-Independent Pixel Synthesis](https://mlsanity.com/paper/2011.13775/) (2020)
- [Generative Models as Distributions of Functions](https://mlsanity.com/paper/2102.04776/) (2021)
- [Hyperspectral pansharpening: a review](https://mlsanity.com/paper/1504.04531/) (2015)
- [A convex formulation for hyperspectral image superresolution via subspace-based regularization](https://mlsanity.com/paper/1411.4005/) (2014)
- [Low-Rank Tensor Function Representation for Multi-Dimensional Data Recovery](https://mlsanity.com/paper/2212.00262/) (2022)
- [Fast Hyperspectral Image Denoising and Inpainting Based on Low-Rank and Sparse Representations](https://mlsanity.com/paper/2103.06842/) (2021)
- [Remote Sensing Image Fusion Based on Two-stream Fusion Network](https://mlsanity.com/paper/1711.02549/) (2017)
- [FastHyMix: Fast and Parameter-free Hyperspectral Image Mixed Noise Removal](https://mlsanity.com/paper/2109.08879/) (2021)
- [Implicit Neural Representation Learning for Hyperspectral Image Super-Resolution](https://mlsanity.com/paper/2112.10541/) (2021)
- [Coupled Convolutional Neural Network with Adaptive Response Function Learning for Unsupervised Hyperspectral Super-Resolution](https://mlsanity.com/paper/2007.14007/) (2020)
- [Revisiting Implicit Neural Representations in Low-Level Vision](https://mlsanity.com/paper/2304.10250/) (2023)
- [Band-wise Hyperspectral Image Pansharpening using CNN Model Propagation](https://mlsanity.com/paper/2311.06510/) (2023)
- [Implicit Neural Feature Fusion Function for Multispectral and Hyperspectral Image Fusion](https://mlsanity.com/paper/2307.07288/) (2023)
- [Positional Embedding-Aware Activations](https://mlsanity.com/paper/2306.15242/) (2023)

## Built on by (most cited first, 8 of 8)

- [Continuous Representation Methods, Theories, and Applications: An Overview and Perspectives](https://mlsanity.com/paper/2505.15222/) (2025)
- [SpectralTrain: A Universal Framework for Hyperspectral Image Classification](https://mlsanity.com/paper/2511.16084/) (2025)
- [Hyperspectral Image Fusion with Spectral-Band and Fusion-Scale Agnosticism](https://mlsanity.com/paper/2602.01681/) (2026)
- [G-ZAP: A Generalizable Zero-Shot Framework for Arbitrary-Scale Pansharpening](https://mlsanity.com/paper/2603.14412/) (2026)
- [GSPan: A Continuous Gaussian Primitive Representation for Arbitrary-Scale Pansharpening](https://mlsanity.com/paper/2606.17722/) (2026)
- [Learning Spatial-Spectral Refinement and Calibrating Complementary Observations for Hyperspectral Image Super-Resolution](https://mlsanity.com/paper/2609.05303/) (2026)
- [Super-Resolving Unseen Hyperspectral Sensors at Any Scale via Spatial Operators](https://mlsanity.com/paper/2609.39926/) (2026)
- [Multimodal Remote Sensing Image Registration: A Comprehensive Review, Challenges and Prospects](https://mlsanity.com/paper/2610.11176/) (2026)

## Abstract (arXiv)

The fusion of a low-resolution hyperspectral image (LR-HSI) with a high-resolution multispectral image (HR-MSI) has emerged as an effective technique for achieving HSI super-resolution (SR). Previous studies have mainly concentrated on estimating the posterior distribution of the latent high-resolution hyperspectral image (HR-HSI), leveraging an appropriate image prior and likelihood computed from the discrepancy between the latent HSI and observed images. Low rankness stands out for preserving latent HSI characteristics through matrix factorization among the various priors. However, the primary limitation in previous studies lies in the generalization of a fusion model with fixed resolution scales, which necessitates retraining whenever output resolutions are changed. To overcome this limitation, we propose a novel continuous low-rank factorization (CLoRF) by integrating two neural representations into the matrix factorization, capturing spatial and spectral information, respectively. This approach enables us to harness both the low rankness from the matrix factorization and the continuity from neural representation in a self-supervised manner.Theoretically, we prove the low-rank property and Lipschitz continuity in the proposed continuous low-rank factorization. Experimentally, our method significantly surpasses existing techniques and achieves user-desired resolutions without the need for neural network retraining. Code is available at https://github.com/wangting1907/CLoRF-Fusion.