# Self-Supervised Image Denoising for Real-World Images with Context-aware Transformer

arXiv: https://arxiv.org/abs/2304.01627 · HTML: https://mlsanity.com/paper/2304.01627/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 53 references that are in the map.
- First posted on arXiv 4 Apr 2023.

## Builds on (most cited first, 30 of 31)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising](https://mlsanity.com/paper/1608.03981/) (2016)
- [Pre-Trained Image Processing Transformer](https://mlsanity.com/paper/2012.00364/) (2020)
- [Noise2Noise: Learning Image Restoration without Clean Data](https://mlsanity.com/paper/1803.04189/) (2018)
- [Noise2Void - Learning Denoising from Single Noisy Images](https://mlsanity.com/paper/1811.10980/) (2018)
- [Noise2Self: Blind Denoising by Self-Supervision](https://mlsanity.com/paper/1901.11365/) (2019)
- [Toward Convolutional Blind Denoising of Real Photographs](https://mlsanity.com/paper/1807.04686/) (2018)
- [Unprocessing Images for Learned Raw Denoising](https://mlsanity.com/paper/1811.11127/) (2018)
- [Real Image Denoising with Feature Attention](https://mlsanity.com/paper/1904.07396/) (2019)
- [Neighbor2Neighbor: Self-Supervised Denoising from Single Noisy Images](https://mlsanity.com/paper/2101.02824/) (2021)
- [High-Quality Self-Supervised Deep Image Denoising](https://mlsanity.com/paper/1901.10277/) (2019)
- [Burst Denoising with Kernel Prediction Networks](https://mlsanity.com/paper/1712.02327/) (2017)
- [Noisier2Noise: Learning to Denoise from Unpaired Noisy Data](https://mlsanity.com/paper/1910.11908/) (2019)
- [Dual Adversarial Network: Toward Real-world Noise Removal and Noise Generation](https://mlsanity.com/paper/2007.05946/) (2020)
- [AP-BSN: Self-Supervised Denoising for Real-World Images via Asymmetric PD and Blind-Spot Network](https://mlsanity.com/paper/2203.11799/) (2022)
- [Transfer Learning from Synthetic to Real-Noise Denoising with Adaptive Instance Normalization](https://mlsanity.com/paper/2002.11244/) (2020)
- [Blind2Unblind: Self-Supervised Image Denoising with Visible Blind Spots](https://mlsanity.com/paper/2203.06967/) (2022)
- [Unpaired Learning of Deep Image Denoising](https://mlsanity.com/paper/2008.13711/) (2020)
- [A Poisson-Gaussian Denoising Dataset with Real Fluorescence Microscopy Images](https://mlsanity.com/paper/1812.10366/) (2018)
- [Ghost-free High Dynamic Range Imaging with Context-aware Transformer](https://mlsanity.com/paper/2208.05114/) (2022)
- [Spatial-Adaptive Network for Single Image Denoising](https://mlsanity.com/paper/2001.10291/) (2020)
- [Multi-level Wavelet Convolutional Neural Networks](https://mlsanity.com/paper/1907.03128/) (2019)
- [Probabilistic Noise2Void: Unsupervised Content-Aware Denoising](https://mlsanity.com/paper/1906.00651/) (2019)
- [CVF-SID: Cyclic multi-Variate Function for Self-Supervised Image Denoising by Disentangling Noise from Image](https://mlsanity.com/paper/2203.13009/) (2022)
- [IDR: Self-Supervised Image Denoising via Iterative Data Refinement](https://mlsanity.com/paper/2111.14358/) (2021)
- [Eformer: Edge Enhancement based Transformer for Medical Image Denoising](https://mlsanity.com/paper/2109.08044/) (2021)
- [Learning Deformable Kernels for Image and Video Denoising](https://mlsanity.com/paper/1904.06903/) (2019)

## Built on by (most cited first, 13 of 13)

- [Solving Inverse Problems with Latent Diffusion Models via Hard Data Consistency](https://mlsanity.com/paper/2307.08123/) (2023)
- [Unleashing the Power of Self-Supervised Image Denoising: A Comprehensive Review](https://mlsanity.com/paper/2308.00247/) (2023)
- [SAM for Poultry Science](https://mlsanity.com/paper/2305.10254/) (2023)
- [High Dynamic Range Imaging with Context-aware Transformer](https://mlsanity.com/paper/2304.04416/) (2023)
- [On-ramp and Off-ramp Traffic Flows Estimation Based on A Data-driven Transfer Learning Framework](https://mlsanity.com/paper/2308.03538/) (2023)
- [A Transfer Learning Framework for Proactive Ramp Metering Performance Assessment](https://mlsanity.com/paper/2308.03542/) (2023)
- [A Brief Overview of Optimization-Based Algorithms for MRI Reconstruction Using Deep Learning](https://mlsanity.com/paper/2406.02626/) (2024)
- [SwinIA: Self-Supervised Blind-Spot Image Denoising without Convolutions](https://mlsanity.com/paper/2305.05651/) (2023)
- [Tell Me What You See: Text-Guided Real-World Image Denoising](https://mlsanity.com/paper/2312.10191/) (2023)
- [Statistically Efficient Variance Reduction with Double Policy Estimation for Off-Policy Evaluation in Sequence-Modeled Reinforcement Learning](https://mlsanity.com/paper/2308.14897/) (2023)
- [Mesh Denoising Transformer](https://mlsanity.com/paper/2405.06536/) (2024)
- [SwinSF: Image Reconstruction from Spatial-Temporal Spike Streams](https://mlsanity.com/paper/2407.15708/) (2024)
- [Deploying Self-Supervised Learning for Real Seismic Data Denoising](https://mlsanity.com/paper/2605.11109/) (2026)

## Abstract (arXiv)

In recent years, the development of deep learning has been pushing image denoising to a new level. Among them, self-supervised denoising is increasingly popular because it does not require any prior knowledge. Most of the existing self-supervised methods are based on convolutional neural networks (CNN), which are restricted by the locality of the receptive field and would cause color shifts or textures loss. In this paper, we propose a novel Denoise Transformer for real-world image denoising, which is mainly constructed with Context-aware Denoise Transformer (CADT) units and Secondary Noise Extractor (SNE) block. CADT is designed as a dual-branch structure, where the global branch uses a window-based Transformer encoder to extract the global information, while the local branch focuses on the extraction of local features with small receptive field. By incorporating CADT as basic components, we build a hierarchical network to directly learn the noise distribution information through residual learning and obtain the first stage denoised output. Then, we design SNE in low computation for secondary global noise extraction. Finally the blind spots are collected from the Denoise Transformer output and reconstructed, forming the final denoised image. Extensive experiments on the real-world SIDD benchmark achieve 50.62/0.990 for PSNR/SSIM, which is competitive with the current state-of-the-art method and only 0.17/0.001 lower. Visual comparisons on public sRGB, Raw-RGB and greyscale datasets prove that our proposed Denoise Transformer has a competitive performance, especially on blurred textures and low-light images, without using additional knowledge, e.g., noise level or noise type, regarding the underlying unknown noise.