mlsanityData to 14 Oct 2026

Feature Statistics with Uncertainty Help Adversarial Robustness

arXiv 2503.20583 ↗First posted 26 Mar 2025cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Despite the remarkable success of deep neural networks (DNNs), the security threat of adversarial attacks poses a significant challenge to the reliability of DNNs. In this paper, both theoretically and empirically, we discover a universal phenomenon that has been neglected in previous works, i.e., adversarial attacks tend to shift the distributions of feature statistics. Motivated by this finding, and by leveraging the advantages of uncertainty-aware stochastic methods in building robust models efficiently, we propose an uncertainty-driven feature statistics adjustment module for robustness enhancement, named Feature Statistics with Uncertainty (FSU). It randomly resamples channel-wise feature means and standard deviations of examples from multivariate Gaussian distributions, which helps to reconstruct the perturbed examples and calibrate the shifted distributions. The calibration recovers some domain characteristics of the data for classification, thereby mitigating the influence of perturbations and weakening the ability of attacks to deceive models. The proposed FSU module has universal applicability in training, attacking, predicting, and fine-tuning, demonstrating impressive robustness enhancement ability at a trivial additional time cost. For example, by fine-tuning the well-established models with FSU, the state-of-the-art methods achieve up to 17.13% and 34.82% robustness improvement against powerful AA and CW attacks on benchmark datasets.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 37 of its 48 references that are in the map. First posted on arXiv 26 Mar 2025.

Builds on

37 of 48 references.

References · 37

  1. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  2. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  3. 2013Intriguing properties of neural networkscited by 5,155
  4. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,998
  5. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,517
  6. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,451
  7. 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,340
  8. 2016Adversarial Machine Learning at Scalecited by 936
  9. 2016Deep Variational Information Bottleneckcited by 930
  10. 2019Square Attack: a query-efficient black-box adversarial attack via random searchcited by 685
  11. 2019Adversarial Training for Free!cited by 643
  12. 2017One pixel attack for fooling deep neural networkscited by 582
25 more
  1. 2017Mitigating Adversarial Effects Through Randomizationcited by 383
  2. 2019Minimally distorted Adversarial Examples with a Fast Adaptive Boundary Attackcited by 334
  3. 2023Better Diffusion Models Further Improve Adversarial Trainingcited by 208
  4. 2018Certified Adversarial Robustness with Additive Noisecited by 180
  5. 2017Towards Robust Neural Networks via Random Self-ensemblecited by 159
  6. 2018Parametric Noise Injection: Trainable Randomness to Improve Deep Neural Network Robustness against Adversarial Attackcited by 125
  7. 2022Uncertainty Modeling for Out-of-Distribution Generalizationcited by 125
  8. 2013Multi-digit Number Recognition from Street View Imagery using Deep Convolutional Neural Networkscited by 84
  9. 2018Adv-BNN: Improved Adversarial Defense through Robust Bayesian Neural Networkcited by 69
  10. 2019advertorch v0.1: An Adversarial Robustness Toolbox based on PyTorchcited by 69
  11. 2023Decoupled Kullback-Leibler Divergence Losscited by 58
  12. 2021Probabilistic Margins for Instance Reweighting in Adversarial Trainingcited by 57
  13. 2019Adversarial Defense by Restricting the Hidden Space of Deep Neural Networkscited by 56
  14. 2022Understanding Robust Overfitting of Adversarial Training and Beyondcited by 51
  15. 2020Robustness of Bayesian Neural Networks to Gradient-Based Attackscited by 48
  16. 2016Simple Black-Box Adversarial Perturbations for Deep Networkscited by 42
  17. 2021GradDiv: Adversarial Robustness of Randomized Neural Networks via Gradient Diversity Regularizationcited by 34
  18. 2020Learn2Perturb: an End-to-end Feature Perturbation Learning to Improve Adversarial Robustnesscited by 27
  19. 2024On the Duality Between Sharpness-Aware Minimization and Adversarial Trainingcited by 24
  20. 2023Improving Generalization of Adversarial Training via Robust Critical Fine-Tuningcited by 23
  21. 2021Adversarial Robustness via Fisher-Rao Regularizationcited by 19
  22. 2023Mitigating Accuracy-Robustness Trade-off via Balanced Multi-Teacher Adversarial Distillationcited by 13
  23. 2020Weight-Covariance Alignment for Adversarially Robust Neural Networkscited by 9
  24. 2023Revisiting the Trade-off between Accuracy and Robustness via Weight Distribution of Filterscited by 4
  25. 2024Robustness-Congruent Adversarial Training for Secure Machine Learning Model Updatescited by 4

Built on by

Not cited yet.