# Unsupervised Spectral Demosaicing with Lightweight Spectral Attention Networks

arXiv: https://arxiv.org/abs/2307.01990 · HTML: https://mlsanity.com/paper/2307.01990/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 64 references that are in the map.
- First posted on arXiv 5 Jul 2023.

## Builds on (most cited first, 19 of 19)

- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network](https://mlsanity.com/paper/1609.05158/) (2016)
- [Deep Image Prior](https://mlsanity.com/paper/1711.10925/) (2017)
- [Plug-and-Play Image Restoration with Deep Denoiser Prior](https://mlsanity.com/paper/2008.13751/) (2020)
- [Equivariant Imaging: Learning Beyond the Range Space](https://mlsanity.com/paper/2103.14756/) (2021)
- [Robust Equivariant Imaging: a fully unsupervised framework for learning to image from noisy and partial measurements](https://mlsanity.com/paper/2111.12855/) (2021)
- [Material Based Object Tracking in Hyperspectral Videos: Benchmark and Algorithms](https://mlsanity.com/paper/1812.04179/) (2018)
- [Deep Plug-and-Play Prior for Hyperspectral Image Restoration](https://mlsanity.com/paper/2209.08240/) (2022)
- [Joint Denoising and Demosaicking with Green Channel Prior for Real-world Burst Images](https://mlsanity.com/paper/2101.09870/) (2021)
- [Deep Plug-and-play Prior for Low-rank Tensor Completion](https://mlsanity.com/paper/1905.04449/) (2019)
- [Rethinking Learning-based Demosaicing, Denoising, and Super-Resolution Pipeline](https://mlsanity.com/paper/1905.02538/) (2019)
- [Deep Learning Approach for Hyperspectral Image Demosaicking, Spectral Correction and High-resolution RGB Reconstruction](https://mlsanity.com/paper/2109.01403/) (2021)
- [Deep demosaicking for multispectral filter arrays](https://mlsanity.com/paper/1808.08021/) (2018)
- [Hyperspectral Demosaicing of Snapshot Camera Images Using Deep Learning](https://mlsanity.com/paper/2211.15435/) (2022)
- [Spatial gradient consistency for unsupervised learning of hyperspectral demosaicking: Application to surgical imaging](https://mlsanity.com/paper/2302.10927/) (2023)
- [Low Cost Edge Sensing for High Quality Demosaicking](https://mlsanity.com/paper/1806.00771/) (2018)
- [Gradient-based Feature Extraction From Raw Bayer Pattern Images](https://mlsanity.com/paper/2004.02429/) (2020)
- [Keynote: Small Neural Nets Are Beautiful: Enabling Embedded Systems with Small Deep-Neural-Network Architectures](https://mlsanity.com/paper/1710.02759/) (2017)
- [Learning Pixel-Adaptive Weights for Portrait Photo Retouching](https://mlsanity.com/paper/2112.03536/) (2021)

## Built on by (most cited first, 5 of 5)

- [Perspective-Equivariance for Unsupervised Imaging with Camera Geometry](https://mlsanity.com/paper/2403.09327/) (2024)
- [A self-supervised and adversarial approach to hyperspectral demosaicking and RGB reconstruction in surgical imaging](https://mlsanity.com/paper/2407.19282/) (2024)
- [Generative Model-Assisted Demosaicing for Cross-multispectral Cameras](https://mlsanity.com/paper/2503.02322/) (2025)
- [Perspective-Equivariant Fine-tuning for Multispectral Demosaicing without Ground Truth](https://mlsanity.com/paper/2603.01332/) (2026)
- [Semi-Supervised Flow Matching for Mosaiced and Panchromatic Fusion Imaging](https://mlsanity.com/paper/2604.20128/) (2026)

## Abstract (arXiv)

This paper presents a deep learning-based spectral demosaicing technique trained in an unsupervised manner. Many existing deep learning-based techniques relying on supervised learning with synthetic images, often underperform on real-world images especially when the number of spectral bands increases. According to the characteristics of the spectral mosaic image, this paper proposes a mosaic loss function, the corresponding model structure, a transformation strategy, and an early stopping strategy, which form a complete unsupervised spectral demosaicing framework. A challenge in real-world spectral demosaicing is inconsistency between the model parameters and the computational resources of the imager. We reduce the complexity and parameters of the spectral attention module by dividing the spectral attention tensor into spectral attention matrices in the spatial dimension and spectral attention vector in the channel dimension, which is more suitable for unsupervised framework. This paper also presents Mosaic25, a real 25-band hyperspectral mosaic image dataset of various objects, illuminations, and materials for benchmarking. Extensive experiments on synthetic and real-world datasets demonstrate that the proposed method outperforms conventional unsupervised methods in terms of spatial distortion suppression, spectral fidelity, robustness, and computational cost.