# Normalization Equivariance for Arbitrary Backbones, with Application to Image Denoising

arXiv: https://arxiv.org/abs/2605.08193 · HTML: https://mlsanity.com/paper/2605.08193/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 37 references that are in the map.
- First posted on arXiv 5 May 2026.

## Builds on (most cited first, 22 of 22)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017)
- [Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising](https://mlsanity.com/paper/1608.03981/) (2016)
- [Instance Normalization: The Missing Ingredient for Fast Stylization](https://mlsanity.com/paper/1607.08022/) (2016)
- [Noise2Noise: Learning Image Restoration without Clean Data](https://mlsanity.com/paper/1803.04189/) (2018)
- [Plug-and-Play Image Restoration with Deep Denoiser Prior](https://mlsanity.com/paper/2008.13751/) (2020)
- [Non-stationary Transformers: Exploring the Stationarity in Time Series Forecasting](https://mlsanity.com/paper/2205.14415/) (2022)
- [Generalization in diffusion models arises from geometry-adaptive harmonic representations](https://mlsanity.com/paper/2310.02557/) (2023)
- [Equivariance with Learned Canonicalization Functions](https://mlsanity.com/paper/2211.06489/) (2022)
- [Robust and interpretable blind image denoising via bias-free convolutional neural networks](https://mlsanity.com/paper/1906.05478/) (2019)
- [RARE: Image Reconstruction using Deep Priors Learned without Ground Truth](https://mlsanity.com/paper/1912.05854/) (2019)
- [Equivariant Adaptation of Large Pretrained Models](https://mlsanity.com/paper/2310.01647/) (2023)
- [Learning normalized image densities via dual score matching](https://mlsanity.com/paper/2506.05310/) (2025)
- [Normalization-Equivariant Neural Networks with Application to Image Denoising](https://mlsanity.com/paper/2306.05037/) (2023)
- [Self-supervised Deep Unrolled Reconstruction Using Regularization by Denoising](https://mlsanity.com/paper/2205.03519/) (2022)
- [Provable Preconditioned Plug-and-Play Approach for Compressed Sensing MRI Reconstruction](https://mlsanity.com/paper/2405.03854/) (2024)
- [Normalization-equivariant Diffusion Models: Learning Posterior Samplers From Noisy And Partial Measurements](https://mlsanity.com/paper/2510.11964/) (2025)
- [On normalization-equivariance properties of supervised and unsupervised denoising methods: a survey](https://mlsanity.com/paper/2402.15352/) (2024)

## Abstract (arXiv)

Normalization Equivariance (NE) is a structural prior that improves robustness to distribution shift in image-to-image tasks. A function f is normalization equivariant iff f(a y + b1) = a f(y) + b1 for all a>0 and b∈ℝ. Existing NE methods constrain every internal layer to NE-compatible operations. These constraints add runtime cost and exclude standard transformer components such as softmax attention and LayerNorm. We introduce Wrapped Normalization Equivariance (WNE), a parameter-free wrapper that normalizes the input, applies any backbone, and denormalizes the output. We prove every NE function admits this factorization, so the wrapper exactly parameterizes the class of NE functions. On blind denoising, wrapping CNN and transformer architectures improves robustness under noise-level mismatch with no measurable GPU overhead, while architectural NE baselines are up to 1.6× slower.