# Positional Normalization

arXiv: https://arxiv.org/abs/1907.04312 · HTML: https://mlsanity.com/paper/1907.04312/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Jul 2019.

## Built on by (most cited first, 28 of 28)

- [Language-driven Semantic Segmentation](https://mlsanity.com/paper/2201.03546/) (2022)
- [Rethinking Spatial Dimensions of Vision Transformers](https://mlsanity.com/paper/2103.16302/) (2021)
- [PD-GAN: Probabilistic Diverse GAN for Image Inpainting](https://mlsanity.com/paper/2105.02201/) (2021)
- [WaveFill: A Wavelet-based Generation Network for Image Inpainting](https://mlsanity.com/paper/2107.11027/) (2021)
- [Unbalanced Feature Transport for Exemplar-based Image Translation](https://mlsanity.com/paper/2106.10482/) (2021)
- [B-cos Alignment for Inherently Interpretable CNNs and Vision Transformers](https://mlsanity.com/paper/2306.10898/) (2023)
- [A Multi-view Multi-task Learning Framework for Multi-variate Time Series Forecasting](https://mlsanity.com/paper/2109.01657/) (2021)
- [A Survey on Mixup Augmentations and Beyond](https://mlsanity.com/paper/2409.05202/) (2024)
- [Adversarial Feature Augmentation and Normalization for Visual Recognition](https://mlsanity.com/paper/2103.12171/) (2021)
- [Underwater Image Enhancement via Learning Water Type Desensitized Representations](https://mlsanity.com/paper/2102.00676/) (2021)
- [Masked and Adaptive Transformer for Exemplar Based Image Translation](https://mlsanity.com/paper/2303.17123/) (2023)
- [Sandwich Batch Normalization: A Drop-In Replacement for Feature Distribution Heterogeneity](https://mlsanity.com/paper/2102.11382/) (2021)
- [Delving into the Estimation Shift of Batch Normalization in a Network](https://mlsanity.com/paper/2203.10778/) (2022)
- [Pyramid Pixel Context Adaption Network for Medical Image Classification with Supervised Contrastive Learning](https://mlsanity.com/paper/2303.01917/) (2023)
- [Flow-based Spatio-Temporal Structured Prediction of Motion Dynamics](https://mlsanity.com/paper/2104.04391/) (2021)
- [Multiscale Residual Learning of Graph Convolutional Sequence Chunks for Human Motion Prediction](https://mlsanity.com/paper/2308.16801/) (2023)
- [ID-Unet: Iterative Soft and Hard Deformation for View Synthesis](https://mlsanity.com/paper/2103.02264/) (2021)
- [SITTA: Single Image Texture Translation for Data Augmentation](https://mlsanity.com/paper/2106.13804/) (2021)
- [Image-to-Image Translation with Low Resolution Conditioning](https://mlsanity.com/paper/2107.11262/) (2021)
- [Interactive Image Inpainting Using Semantic Guidance](https://mlsanity.com/paper/2201.10753/) (2022)
- [Kernel Normalized Convolutional Networks](https://mlsanity.com/paper/2205.10089/) (2022)
- [DSI2I: Dense Style for Unpaired Image-to-Image Translation](https://mlsanity.com/paper/2212.13253/) (2022)
- [L2BN: Enhancing Batch Normalization by Equalizing the L2 Norms of Features](https://mlsanity.com/paper/2207.02625/) (2022)
- [Domain Agnostic Image-to-image Translation using Low-Resolution Conditioning](https://mlsanity.com/paper/2305.05023/) (2023)
- [Unified Batch Normalization: Identifying and Alleviating the Feature Condensation in Batch Normalization and a Unified Framework](https://mlsanity.com/paper/2311.15993/) (2023)
- [Decoupling Degradation and Content Processing for Adverse Weather Image Restoration](https://mlsanity.com/paper/2312.05006/) (2023)
- [Robust Calibration For Improved Weather Prediction Under Distributional Shift](https://mlsanity.com/paper/2401.04144/) (2024)
- [YOLO-RD: Introducing Relevant and Compact Explicit Knowledge to YOLO by Retriever-Dictionary](https://mlsanity.com/paper/2410.15346/) (2024)

## Abstract (arXiv)

A popular method to reduce the training time of deep neural networks is to normalize activations at each layer. Although various normalization schemes have been proposed, they all follow a common theme: normalize across spatial dimensions and discard the extracted statistics. In this paper, we propose an alternative normalization method that noticeably departs from this convention and normalizes exclusively across channels. We argue that the channel dimension is naturally appealing as it allows us to extract the first and second moments of features extracted at a particular image position. These moments capture structural information about the input image and extracted features, which opens a new avenue along which a network can benefit from feature normalization: Instead of disregarding the normalization constants, we propose to re-inject them into later layers to preserve or transfer structural information in generative networks. Codes are available at https://github.com/Boyiliee/PONO.