# Batch Normalization Increases Adversarial Vulnerability and Decreases Adversarial Transferability: A Non-Robust Feature Perspective

arXiv: https://arxiv.org/abs/2010.03316 · HTML: https://mlsanity.com/paper/2010.03316/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Oct 2020.

## Built on by (most cited first, 30 of 35)

- [A Survey on Transferability of Adversarial Examples across Deep Neural Networks](https://mlsanity.com/paper/2310.17626/) (2023)
- [Neural Mean Discrepancy for Efficient Out-of-Distribution Detection](https://mlsanity.com/paper/2104.11408/) (2021)
- [SA-Attack: Improving Adversarial Transferability of Vision-Language Pre-training Models via Self-Augmentation](https://mlsanity.com/paper/2312.04913/) (2023)
- [Normalization Layers Are All That Sharpness-Aware Minimization Needs](https://mlsanity.com/paper/2306.04226/) (2023)
- [Investigating Top-k White-Box and Transferable Black-box Attack](https://mlsanity.com/paper/2204.00089/) (2022)
- [Removing Batch Normalization Boosts Adversarial Training](https://mlsanity.com/paper/2207.01156/) (2022)
- [Transferable Adversarial Attacks on SAM and Its Downstream Models](https://mlsanity.com/paper/2410.20197/) (2024)
- [Adversarially-Aware Robust Object Detector](https://mlsanity.com/paper/2207.06202/) (2022)
- [Feature Separation and Recalibration for Adversarial Robustness](https://mlsanity.com/paper/2303.13846/) (2023)
- [Adversarial Examples Are Not Real Features](https://mlsanity.com/paper/2310.18936/) (2023)
- [Reliable Evaluation of Adversarial Transferability](https://mlsanity.com/paper/2306.08565/) (2023)
- [Breaking Barriers in Physical-World Adversarial Examples: Improving Robustness and Transferability via Robust Feature](https://mlsanity.com/paper/2412.16958/) (2024)
- [StyLess: Boosting the Transferability of Adversarial Examples](https://mlsanity.com/paper/2304.11579/) (2023)
- [Improving Adversarial Transferability via Model Alignment](https://mlsanity.com/paper/2311.18495/) (2023)
- [On the Role of Generalization in Transferability of Adversarial Examples](https://mlsanity.com/paper/2206.09238/) (2022)
- [Understanding the robustness difference between stochastic gradient descent and adaptive gradient methods](https://mlsanity.com/paper/2308.06703/) (2023)
- [Adversarially Robust Spiking Neural Networks Through Conversion](https://mlsanity.com/paper/2311.09266/) (2023)
- [Exploring Adversarial Transferability between Kolmogorov-arnold Networks](https://mlsanity.com/paper/2503.06276/) (2025)
- [Functional Network: A Novel Framework for Interpretability of Deep Neural Networks](https://mlsanity.com/paper/2205.11702/) (2022)
- [NoMorelization: Building Normalizer-Free Models from a Sample's Perspective](https://mlsanity.com/paper/2210.06932/) (2022)
- [Going Further: Flatness at the Rescue of Early Stopping for Adversarial Example Transferability](https://mlsanity.com/paper/2304.02688/) (2023)
- [Theoretical Understanding of Learning from Adversarial Perturbations](https://mlsanity.com/paper/2402.10470/) (2024)
- [Test-Time Defense Against Adversarial Attacks via Stochastic Resonance of Latent Ensembles](https://mlsanity.com/paper/2510.03224/) (2025)
- [Towards Bridging the gap between Empirical and Certified Robustness against Adversarial Examples](https://mlsanity.com/paper/2102.05096/) (2021)
- [Improved Robustness Against Adaptive Attacks With Ensembles and Error-Correcting Output Codes](https://mlsanity.com/paper/2303.02322/) (2023)
- [RAMP: Boosting Adversarial Robustness Against Multiple lp Perturbations for Universal Robustness](https://mlsanity.com/paper/2402.06827/) (2024)
- [Investigating and unmasking feature-level vulnerabilities of CNNs to adversarial perturbations](https://mlsanity.com/paper/2405.20672/) (2024)
- [DisCoPatch: Taming Adversarially-driven Batch Statistics for Improved Out-of-Distribution Detection](https://mlsanity.com/paper/2501.08005/) (2025)
- [Batch Normalization Amplifies Memorization and Privacy Risks](https://mlsanity.com/paper/2605.24420/) (2026)
- [L2BN: Enhancing Batch Normalization by Equalizing the L2 Norms of Features](https://mlsanity.com/paper/2207.02625/) (2022)

## Abstract (arXiv)

Batch normalization (BN) has been widely used in modern deep neural networks (DNNs) due to improved convergence. BN is observed to increase the model accuracy while at the cost of adversarial robustness. There is an increasing interest in the ML community to understand the impact of BN on DNNs, especially related to the model robustness. This work attempts to understand the impact of BN on DNNs from a non-robust feature perspective. Straightforwardly, the improved accuracy can be attributed to the better utilization of useful features. It remains unclear whether BN mainly favors learning robust features (RFs) or non-robust features (NRFs). Our work presents empirical evidence that supports that BN shifts a model towards being more dependent on NRFs. To facilitate the analysis of such a feature robustness shift, we propose a framework for disentangling robust usefulness into robustness and usefulness. Extensive analysis under the proposed framework yields valuable insight on the DNN behavior regarding robustness, e.g. DNNs first mainly learn RFs and then NRFs. The insight that RFs transfer better than NRFs, further inspires simple techniques to strengthen transfer-based black-box attacks.