Perspective-Equivariant Fine-tuning for Multispectral Demosaicing without Ground Truth
Not on the map. Why?
Abstract (arXiv)
Multispectral demosaicing is crucial to reconstruct full-resolution spectral images from snapshot mosaiced measurements, enabling real-time imaging from neurosurgery to autonomous driving. Classical methods are blurry, while supervised learning requires costly ground truth (GT) obtained from slow line-scanning systems. We propose Perspective-Equivariant Fine-tuning for Demosaicing (PEFD), a framework that learns multispectral demosaicing from mosaiced measurements alone. PEFD a) exploits the projective geometry of camera-based imaging systems to leverage a richer group structure than previous demosaicing methods to recover more null-space information, and b) learns efficiently without GT by adapting pretrained foundation models designed for 1-3 channel imaging. On surgical and automotive datasets, PEFD recovers fine details such as blood vessels and preserves spectral fidelity, substantially outperforming recent approaches, nearing supervised performance. Furthermore, the performance of PEFD is demonstrated on raw, unprocessed data from a commercial multispectral sensor. Code is at https://github.com/Andrewwango/pefd.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 26 of its 57 references that are in the map. First posted on arXiv 2 Mar 2026.
Builds on
26 of 57 references.
References · 26
- 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
- 2022Generative Adversarial Networkscited by 5,603
- 2017Enhanced Deep Residual Networks for Single Image Super-Resolutioncited by 1,543
- 2016Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoisingcited by 1,352
- 2017Deep Image Priorcited by 1,097
- 2020Plug-and-Play Image Restoration with Deep Denoiser Priorcited by 467
- 2018Noise2Void - Learning Denoising from Single Noisy Imagescited by 359
- 2021Neighbor2Neighbor: Self-Supervised Denoising from Single Noisy Imagescited by 171
- 2021Equivariant Imaging: Learning Beyond the Range Spacecited by 83
- 2021Robust Equivariant Imaging: a fully unsupervised framework for learning to image from noisy and partial measurementscited by 56
- 2022Test-Time Training Can Close the Natural Distribution Shift Performance Gap in Deep Learning Based Compressed Sensingcited by 37
- 2025DeepInverse: A Python package for solving imaging inverse problems with deep learningcited by 27
14 more
- 2022Deep Plug-and-Play Prior for Hyperspectral Image Restorationcited by 22
- 2024Generalized Recorrupted-to-Recorrupted: Self-Supervised Learning Beyond Gaussian Noisecited by 13
- 2025PIDSR: Complementary Polarized Image Demosaicing and Super-Resolutioncited by 11
- 2022Clean self-supervised MRI reconstruction from noisy, sub-sampled training data with Robust SSDUcited by 10
- 2023Hyperspectral Image Segmentation: A Preliminary Study on the Oral and Dental Spectral Image Database (ODSI-DB)cited by 10
- 2023Unsupervised Deep Learning-based Pansharpening with Jointly-Enhanced Spectral and Spatial Fidelitycited by 8
- 2021Deep Learning Approach for Hyperspectral Image Demosaicking, Spectral Correction and High-resolution RGB Reconstructioncited by 7
- 2024In-Vivo Hyperspectral Human Brain Image Database for Brain Cancer Detectioncited by 6
- 2023Computational Spectral Imaging: A Contemporary Overviewcited by 5
- 2023Unsupervised Spectral Demosaicing with Lightweight Spectral Attention Networkscited by 5
- 2024Perspective-Equivariance for Unsupervised Imaging with Camera Geometrycited by 5
- 2023Spatial gradient consistency for unsupervised learning of hyperspectral demosaicking: Application to surgical imagingcited by 4
- 2024A self-supervised and adversarial approach to hyperspectral demosaicking and RGB reconstruction in surgical imagingcited by 4
- 2023Joint Demosaicing and Denoising with Double Deep Image Priorscited by 3
Built on by
Not cited yet.