# SGDFormer: One-stage Transformer-based Architecture for Cross-Spectral Stereo Image Guided Denoising

arXiv: https://arxiv.org/abs/2404.00349 · HTML: https://mlsanity.com/paper/2404.00349/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 62 references that are in the map.
- First posted on arXiv 30 Mar 2024.

## Builds on (most cited first, 26 of 26)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising](https://mlsanity.com/paper/1608.03981/) (2016)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [Pre-Trained Image Processing Transformer](https://mlsanity.com/paper/2012.00364/) (2020)
- [A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation](https://mlsanity.com/paper/1512.02134/) (2015)
- [Uformer: A General U-Shaped Transformer for Image Restoration](https://mlsanity.com/paper/2106.03106/) (2021)
- [Multi-Stage Progressive Image Restoration](https://mlsanity.com/paper/2102.02808/) (2021)
- [Simple Baselines for Image Restoration](https://mlsanity.com/paper/2204.04676/) (2022)
- [Unifying Flow, Stereo and Depth Estimation](https://mlsanity.com/paper/2211.05783/) (2022)
- [Neighborhood Attention Transformer](https://mlsanity.com/paper/2204.07143/) (2022)
- [HINet: Half Instance Normalization Network for Image Restoration](https://mlsanity.com/paper/2105.06086/) (2021)
- [Deep Generalized Unfolding Networks for Image Restoration](https://mlsanity.com/paper/2204.13348/) (2022)
- [Deep Convolutional Neural Network for Multi-modal Image Restoration and Fusion](https://mlsanity.com/paper/1910.04066/) (2019)
- [NAFSSR: Stereo Image Super-Resolution Using NAFNet](https://mlsanity.com/paper/2204.08714/) (2022)
- [Learning Parallax Attention for Stereo Image Super-Resolution](https://mlsanity.com/paper/1903.05784/) (2019)
- [DAVANet: Stereo Deblurring with View Aggregation](https://mlsanity.com/paper/1904.05065/) (2019)
- [DarkVisionNet: Low-Light Imaging via RGB-NIR Fusion with Deep Inconsistency Prior](https://mlsanity.com/paper/2303.06834/) (2023)
- [Stereoscopic Dark Flash for Low-light Photography](https://mlsanity.com/paper/1901.01370/) (2019)
- [A cross Transformer for image denoising](https://mlsanity.com/paper/2310.10408/) (2023)
- [An Integrated Enhancement Solution for 24-hour Colorful Imaging](https://mlsanity.com/paper/2005.04580/) (2020)
- [A Dual Sensor Computational Camera for High Quality Dark Videography](https://mlsanity.com/paper/2204.04987/) (2022)

## Built on by (most cited first, 2 of 2)

- [Toward Robust and 3D-Aware RGB-NIR Imaging in the Dark](https://mlsanity.com/paper/2607.29684/) (2026)
- [Rethinking Unsupervised Cross-modal Flow Estimation: Learning from Decoupled Optimization and Consistency Constraint](https://mlsanity.com/paper/2509.24423/) (2025)

## Abstract (arXiv)

Cross-spectral image guided denoising has shown its great potential in recovering clean images with rich details, such as using the near-infrared image to guide the denoising process of the visible one. To obtain such image pairs, a feasible and economical way is to employ a stereo system, which is widely used on mobile devices. Current works attempt to generate an aligned guidance image to handle the disparity between two images. However, due to occlusion, spectral differences and noise degradation, the aligned guidance image generally exists ghosting and artifacts, leading to an unsatisfactory denoised result. To address this issue, we propose a one-stage transformer-based architecture, named SGDFormer, for cross-spectral Stereo image Guided Denoising. The architecture integrates the correspondence modeling and feature fusion of stereo images into a unified network. Our transformer block contains a noise-robust cross-attention (NRCA) module and a spatially variant feature fusion (SVFF) module. The NRCA module captures the long-range correspondence of two images in a coarse-to-fine manner to alleviate the interference of noise. The SVFF module further enhances salient structures and suppresses harmful artifacts through dynamically selecting useful information. Thanks to the above design, our SGDFormer can restore artifact-free images with fine structures, and achieves state-of-the-art performance on various datasets. Additionally, our SGDFormer can be extended to handle other unaligned cross-model guided restoration tasks such as guided depth super-resolution.