# Perspective-Equivariant Fine-tuning for Multispectral Demosaicing without Ground Truth

arXiv: https://arxiv.org/abs/2603.01332 · HTML: https://mlsanity.com/paper/2603.01332/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 26 of its 57 references that are in the map.
- First posted on arXiv 2 Mar 2026.

## Builds on (most cited first, 26 of 26)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017)
- [Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising](https://mlsanity.com/paper/1608.03981/) (2016)
- [Deep Image Prior](https://mlsanity.com/paper/1711.10925/) (2017)
- [Plug-and-Play Image Restoration with Deep Denoiser Prior](https://mlsanity.com/paper/2008.13751/) (2020)
- [Noise2Void - Learning Denoising from Single Noisy Images](https://mlsanity.com/paper/1811.10980/) (2018)
- [Neighbor2Neighbor: Self-Supervised Denoising from Single Noisy Images](https://mlsanity.com/paper/2101.02824/) (2021)
- [Equivariant Imaging: Learning Beyond the Range Space](https://mlsanity.com/paper/2103.14756/) (2021)
- [Robust Equivariant Imaging: a fully unsupervised framework for learning to image from noisy and partial measurements](https://mlsanity.com/paper/2111.12855/) (2021)
- [Test-Time Training Can Close the Natural Distribution Shift Performance Gap in Deep Learning Based Compressed Sensing](https://mlsanity.com/paper/2204.07204/) (2022)
- [DeepInverse: A Python package for solving imaging inverse problems with deep learning](https://mlsanity.com/paper/2505.20160/) (2025)
- [Deep Plug-and-Play Prior for Hyperspectral Image Restoration](https://mlsanity.com/paper/2209.08240/) (2022)
- [Generalized Recorrupted-to-Recorrupted: Self-Supervised Learning Beyond Gaussian Noise](https://mlsanity.com/paper/2412.04648/) (2024)
- [PIDSR: Complementary Polarized Image Demosaicing and Super-Resolution](https://mlsanity.com/paper/2504.07758/) (2025)
- [Clean self-supervised MRI reconstruction from noisy, sub-sampled training data with Robust SSDU](https://mlsanity.com/paper/2210.01696/) (2022)
- [Hyperspectral Image Segmentation: A Preliminary Study on the Oral and Dental Spectral Image Database (ODSI-DB)](https://mlsanity.com/paper/2303.08252/) (2023)
- [Unsupervised Deep Learning-based Pansharpening with Jointly-Enhanced Spectral and Spatial Fidelity](https://mlsanity.com/paper/2307.14403/) (2023)
- [Deep Learning Approach for Hyperspectral Image Demosaicking, Spectral Correction and High-resolution RGB Reconstruction](https://mlsanity.com/paper/2109.01403/) (2021)
- [In-Vivo Hyperspectral Human Brain Image Database for Brain Cancer Detection](https://mlsanity.com/paper/2402.10776/) (2024)
- [Computational Spectral Imaging: A Contemporary Overview](https://mlsanity.com/paper/2303.04848/) (2023)
- [Unsupervised Spectral Demosaicing with Lightweight Spectral Attention Networks](https://mlsanity.com/paper/2307.01990/) (2023)
- [Perspective-Equivariance for Unsupervised Imaging with Camera Geometry](https://mlsanity.com/paper/2403.09327/) (2024)
- [Spatial gradient consistency for unsupervised learning of hyperspectral demosaicking: Application to surgical imaging](https://mlsanity.com/paper/2302.10927/) (2023)
- [A self-supervised and adversarial approach to hyperspectral demosaicking and RGB reconstruction in surgical imaging](https://mlsanity.com/paper/2407.19282/) (2024)
- [Joint Demosaicing and Denoising with Double Deep Image Priors](https://mlsanity.com/paper/2309.09426/) (2023)

## Abstract (arXiv)

Multispectral demosaicing is crucial to reconstruct full-resolution spectral images from snapshot mosaiced measurements, enabling real-time imaging from neurosurgery to autonomous driving. Classical methods are blurry, while supervised learning requires costly ground truth (GT) obtained from slow line-scanning systems. We propose Perspective-Equivariant Fine-tuning for Demosaicing (PEFD), a framework that learns multispectral demosaicing from mosaiced measurements alone. PEFD a) exploits the projective geometry of camera-based imaging systems to leverage a richer group structure than previous demosaicing methods to recover more null-space information, and b) learns efficiently without GT by adapting pretrained foundation models designed for 1-3 channel imaging. On surgical and automotive datasets, PEFD recovers fine details such as blood vessels and preserves spectral fidelity, substantially outperforming recent approaches, nearing supervised performance. Furthermore, the performance of PEFD is demonstrated on raw, unprocessed data from a commercial multispectral sensor. Code is at https://github.com/Andrewwango/pefd.