mlsanityData to 7 Oct 2026

Batch Normalization is a Cause of Adversarial Vulnerability

arXiv 1905.02161 ↗First posted 6 May 2019cs.LG stat.MLcited by 28

Not on the map. Why?

Abstract (arXiv)

Batch normalization (batch norm) is often used in an attempt to stabilize and accelerate training in deep neural networks. In many cases it indeed decreases the number of parameter updates required to achieve low training error. However, it also reduces robustness to small adversarial input perturbations and noise by double-digit percentages, as we show on five standard datasets. Furthermore, substituting weight decay for batch norm is sufficient to nullify the relationship between adversarial vulnerability and the input dimension. Our work is consistent with a mean-field analysis that found that batch norm causes exploding gradients.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). First posted on arXiv 6 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 13; 2022: 6; 2023: 1; 2024: 5; 2025: 2; 2026: 1.

28 papers.

Sort:

By area:

  1. 2021The Norm Must Go On: Dynamic Unsupervised Domain Adaptation by Normalizationcited by 131
  2. 2021Parameter Prediction for Unseen Deep Architecturescited by 89
  3. 2021Improving Adversarial Robustness via Channel-wise Activation Suppressingcited by 76
  4. 2021When Does Contrastive Learning Preserve Adversarial Robustness from Pretraining to Finetuning?cited by 76
  5. 2021Beyond BatchNorm: Towards a Unified Understanding of Normalization in Deep Learningcited by 24
  6. 2022Removing Batch Normalization Boosts Adversarial Trainingcited by 24
  7. 2021Shift Invariance Can Reduce Adversarial Robustnesscited by 16
  8. 2023Feature Separation and Recalibration for Adversarial Robustnesscited by 16
  9. 2021Proxy-Normalizing Activations to Match Batch Normalization while Removing Batch Dependencecited by 12
  10. 2021What Do Deep Nets Learn? Class-wise Patterns Revealed in the Input Spacecited by 8
  11. 2024Domain-Specific Block Selection and Paired-View Pseudo-Labeling for Online Test-Time Adaptationcited by 8
  12. 2024Spectrum Extraction and Clipping for Implicitly Linear Layerscited by 7
16 more
  1. 2021Random and Adversarial Bit Error Robustness: Energy-Efficient and Secure DNN Acceleratorscited by 6
  2. 2021Bridging the Gap Between Adversarial Robustness and Optimization Biascited by 5
  3. 2021Batch Normalization Preconditioning for Neural Network Trainingcited by 5
  4. 2022Fairness Increases Adversarial Vulnerabilitycited by 5
  5. 2022On Fragile Features and Batch Normalization in Adversarial Trainingcited by 4
  6. 2021Clustering Effect of (Linearized) Adversarial Robust Modelscited by 3
  7. 2022Batch Normalization Is Blind to the First and Second Derivatives of the Losscited by 3
  8. 2024Modeling Structured Data Learning with Restricted Boltzmann Machines in the Teacher-Student Settingcited by 2
  9. 2024CausAdv: A Causal-based Framework for Detecting Adversarial Examplescited by 2
  10. 2025Test-Time Defense Against Adversarial Attacks via Stochastic Resonance of Latent Ensemblescited by 2
  11. 2024Investigating and unmasking feature-level vulnerabilities of CNNs to adversarial perturbationscited by 1
  12. 2026Batch Normalization Amplifies Memorization and Privacy Riskscited by 1
  13. 2021Dealing with Adversarial Player Strategies in the Neural Network Game iNNk through Ensemble Learningcited by 0
  14. 2022Counterbalancing Teacher: Regularizing Batch Normalized Models for Robustnesscited by 0
  15. 2022Pitfalls of Conditional Batch Normalization for Contextual Multi-Modal Learningcited by 0
  16. 2025SOLAR: Switchable Output Layer for Accuracy and Robustness in Once-for-All Trainingcited by 0