mlsanityData to 14 Oct 2026

Tuning for Two Adversaries: Enhancing the Robustness Against Transfer and Query-Based Attacks using Hyperparameter Tuning

arXiv 2511.13654 ↗First posted 17 Nov 2025cs.LG cs.CR cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

In this paper, we present the first detailed analysis of how training hyperparameters -- such as learning rate, weight decay, momentum, and batch size -- influence robustness against both transfer-based and query-based attacks. Supported by theory and experiments, our study spans a variety of practical deployment settings, including centralized training, ensemble learning, and distributed training. We uncover a striking dichotomy: for transfer-based attacks, decreasing the learning rate significantly enhances robustness by up to 64%. In contrast, for query-based attacks, increasing the learning rate consistently leads to improved robustness by up to 28% across various settings and data distributions. Leveraging these findings, we explore -- for the first time -- the training hyperparameter space to jointly enhance robustness against both transfer-based and query-based attacks. Our results reveal that distributed models benefit the most from hyperparameter tuning, achieving a remarkable tradeoff by simultaneously mitigating both attack types more effectively than other training setups.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 27 of its 42 references that are in the map. First posted on arXiv 17 Nov 2025.

Builds on

27 of 42 references.

References · 27

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  3. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  4. 2013Intriguing properties of neural networkscited by 5,155
  5. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,451
  6. 2017Ensemble Adversarial Training: Attacks and Defensescited by 840
  7. 2019Square Attack: a query-efficient black-box adversarial attack via random searchcited by 685
  8. 2017ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Modelscited by 666
  9. 2017Countering Adversarial Images using Input Transformationscited by 520
  10. 2018Black-box Adversarial Attacks with Limited Queries and Informationcited by 501
  11. 2021Gradient Descent on Neural Networks Typically Occurs at the Edge of Stabilitycited by 427
  12. 2019HopSkipJumpAttack: A Query-Efficient Decision-Based Attackcited by 333
15 more
  1. 2020Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examplescited by 278
  2. 2017Three Factors Influencing Minima in SGDcited by 273
  3. 2019Improving Adversarial Robustness via Promoting Ensemble Diversitycited by 213
  4. 2021Fixing Data Augmentation to Improve Adversarial Robustnesscited by 204
  5. 2018Robustness via curvature regularization, and vice versacited by 175
  6. 2018AutoZOOM: Autoencoder-based Zeroth Order Optimization Method for Attacking Black-box Neural Networkscited by 137
  7. 2020DVERGE: Diversifying Vulnerabilities for Enhanced Robust Generation of Ensemblescited by 73
  8. 2023Why Do We Need Weight Decay in Modern Deep Learning?cited by 72
  9. 2019Improving Adversarial Robustness of Ensembles with Diversity Trainingcited by 69
  10. 2021On Linear Stability of SGD and Input-Smoothness of Neural Networkscited by 63
  11. 2022On the Maximum Hessian Eigenvalue and Generalizationcited by 57
  12. 2023Understanding and Improving Ensemble Adversarial Defensecited by 14
  13. 2022Adversarial Ensemble Training by Jointly Learning Label Dependencies and Member Modelscited by 2
  14. 2025Two is Better than One: Efficient Ensemble Defense for Robust and Compact Modelscited by 1
  15. 2025On Hyperparameters and Backdoor-Resistance in Horizontal Federated Learningcited by 1

Built on by

Not cited yet.