# Revisiting Batch Normalization for Improving Corruption Robustness

arXiv: https://arxiv.org/abs/2010.03630 · HTML: https://mlsanity.com/paper/2010.03630/

## Key facts

- Cited by 55 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Oct 2020.

## Built on by (most cited first, 30 of 55)

- [A Comprehensive Survey on Test-Time Adaptation under Distribution Shifts](https://mlsanity.com/paper/2303.15361/) (2023)
- [Uncertainty Modeling for Out-of-Distribution Generalization](https://mlsanity.com/paper/2202.03958/) (2022)
- [Test-Time Adaptation to Distribution Shift by Confidence Maximization and Input Transformation](https://mlsanity.com/paper/2106.14999/) (2021)
- [Rethinking "Batch" in BatchNorm](https://mlsanity.com/paper/2105.07576/) (2021)
- [Partial and Asymmetric Contrastive Learning for Out-of-Distribution Detection in Long-Tailed Recognition](https://mlsanity.com/paper/2207.01160/) (2022)
- [PRIME: A few primitives can boost robustness to common corruptions](https://mlsanity.com/paper/2112.13547/) (2021)
- [Robustness of SAM: Segment Anything Under Corruptions and Beyond](https://mlsanity.com/paper/2306.07713/) (2023)
- [Removing Batch Normalization Boosts Adversarial Training](https://mlsanity.com/paper/2207.01156/) (2022)
- [A Survey on the Robustness of Computer Vision Models against Common Corruptions](https://mlsanity.com/paper/2305.06024/) (2023)
- [Universal Adversarial Perturbations Through the Lens of Deep Steganography: Towards A Fourier Perspective](https://mlsanity.com/paper/2102.06479/) (2021)
- [EAT: Towards Long-Tailed Out-of-Distribution Detection](https://mlsanity.com/paper/2312.08939/) (2023)
- [Universal Adversarial Training with Class-Wise Perturbations](https://mlsanity.com/paper/2104.03000/) (2021)
- [Improving Robustness by Enhancing Weak Subnets](https://mlsanity.com/paper/2201.12765/) (2022)
- [Understanding the Failure of Batch Normalization for Transformers in NLP](https://mlsanity.com/paper/2210.05153/) (2022)
- [HybridAugment++: Unified Frequency Spectra Perturbations for Model Robustness](https://mlsanity.com/paper/2307.11823/) (2023)
- [Benchmarking the Robustness of Temporal Action Detection Models Against Temporal Corruptions](https://mlsanity.com/paper/2403.20254/) (2024)
- [Anti-aliasing Deep Image Classifiers using Novel Depth Adaptive Blurring and Activation Function](https://mlsanity.com/paper/2110.00899/) (2021)
- [StyLess: Boosting the Transferability of Adversarial Examples](https://mlsanity.com/paper/2304.11579/) (2023)
- [Test-time Adaptation for Regression by Subspace Alignment](https://mlsanity.com/paper/2410.03263/) (2024)
- [Test-time Batch Normalization](https://mlsanity.com/paper/2205.10210/) (2022)
- [Understanding the robustness difference between stochastic gradient descent and adaptive gradient methods](https://mlsanity.com/paper/2308.06703/) (2023)
- [Domain-Specific Block Selection and Paired-View Pseudo-Labeling for Online Test-Time Adaptation](https://mlsanity.com/paper/2404.10966/) (2024)
- [Delving into the Estimation Shift of Batch Normalization in a Network](https://mlsanity.com/paper/2203.10778/) (2022)
- [REALM: Robust Entropy Adaptive Loss Minimization for Improved Single-Sample Test-Time Adaptation](https://mlsanity.com/paper/2309.03964/) (2023)
- [Un-Mixing Test-Time Normalization Statistics: Combatting Label Temporal Correlation](https://mlsanity.com/paper/2401.08328/) (2024)
- [Spectrum Extraction and Clipping for Implicitly Linear Layers](https://mlsanity.com/paper/2402.16017/) (2024)
- [Improving robustness to corruptions with multiplicative weight perturbations](https://mlsanity.com/paper/2406.16540/) (2024)
- [How Robust are Discriminatively Trained Zero-Shot Learning Models?](https://mlsanity.com/paper/2201.10972/) (2022)
- [Robustness to corruption in pre-trained Bayesian neural networks](https://mlsanity.com/paper/2206.12361/) (2022)
- [Benchmarking performance of object detection under image distortions in an uncontrolled environment](https://mlsanity.com/paper/2210.15999/) (2022)

## Abstract (arXiv)

The performance of DNNs trained on clean images has been shown to decrease when the test images have common corruptions. In this work, we interpret corruption robustness as a domain shift and propose to rectify batch normalization (BN) statistics for improving model robustness. This is motivated by perceiving the shift from the clean domain to the corruption domain as a style shift that is represented by the BN statistics. We find that simply estimating and adapting the BN statistics on a few (32 for instance) representation samples, without retraining the model, improves the corruption robustness by a large margin on several benchmark datasets with a wide range of model architectures. For example, on ImageNet-C, statistics adaptation improves the top1 accuracy of ResNet50 from 39.2% to 48.7%. Moreover, we find that this technique can further improve state-of-the-art robust models from 58.1% to 63.3%.