mlsanityData to 14 Oct 2026

Learning Diverse-Structured Networks for Adversarial Robustness

arXiv 2102.01886 ↗First posted 3 Feb 2021cs.LG cs.CVcited by 12

Not on the map. Why?

Abstract (arXiv)

In adversarial training (AT), the main focus has been the objective and optimizer while the model has been less studied, so that the models being used are still those classic ones in standard training (ST). Classic network architectures (NAs) are generally worse than searched NAs in ST, which should be the same in AT. In this paper, we argue that NA and AT cannot be handled independently, since given a dataset, the optimal NA in ST would be no longer optimal in AT. That being said, AT is time-consuming itself; if we directly search NAs in AT over large search spaces, the computation will be practically infeasible. Thus, we propose a diverse-structured network (DS-Net), to significantly reduce the size of the search space: instead of low-level operations, we only consider predefined atomic blocks, where an atomic block is a time-tested building block like the residual block. There are only a few atomic blocks and thus we can weight all atomic blocks rather than find the best one in a searched block of DS-Net, which is an essential trade-off between exploring diverse structures and exploiting the best structures. Empirical results demonstrate the advantages of DS-Net, i.e., weighting the atomic blocks.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 62 of its 76 references that are in the map. First posted on arXiv 3 Feb 2021.

Builds on

62 of 76 references.

References · 62

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  4. 2016Densely Connected Convolutional Networkscited by 6,720
  5. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  6. 2017Squeeze-and-Excitation Networkscited by 4,789
  7. 2016Wide Residual Networkscited by 3,070
  8. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,517
  9. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,451
  10. 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,340
  11. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,268
  12. 2017Visualizing the Loss Landscape of Neural Netscited by 1,130
38 more
  1. 2018MnasNet: Platform-Aware Neural Architecture Search for Mobilecited by 943
  2. 2018A Convergence Theory for Deep Learning via Over-Parameterizationcited by 787
  3. 2020Fast is better than free: Revisiting adversarial trainingcited by 727
  4. 2013Stochastic First- and Zeroth-order Methods for Nonconvex Stochastic Programmingcited by 724
  5. 2018Gradient Descent Finds Global Minima of Deep Neural Networkscited by 574
  6. 2017Provable defenses against adversarial examples via the convex outer adversarial polytopecited by 567
  7. 2017Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methodscited by 546
  8. 2020Overfitting in adversarially robust deep learningcited by 506
  9. 2017Mitigating Adversarial Effects Through Randomizationcited by 383
  10. 2017Parseval Networks: Improving Robustness to Adversarial Examplescited by 293
  11. 2020Attacks Which Do Not Kill Training Make Adversarial Learning Strongercited by 243
  12. 2019Improving Adversarial Robustness via Promoting Ensemble Diversitycited by 213
  13. 2019Towards Stable and Efficient Training of Verifiably Robust Neural Networkscited by 205
  14. 2020Bag of Tricks for Adversarial Trainingcited by 200
  15. 2016A study of the effect of JPG compression on adversarial imagescited by 197
  16. 2018Lipschitz-Margin Training: Scalable Certification of Perturbation Invariance for Deep Neural Networkscited by 195
  17. 2017Formal Guarantees on the Robustness of a Classifier against Adversarial Manipulationcited by 185
  18. 2019You Only Propagate Once: Accelerating Adversarial Training via Maximal Principlecited by 183
  19. 2018Evaluating the Robustness of Neural Networks: An Extreme Value Theory Approachcited by 182
  20. 2020A Self-supervised Approach for Adversarial Robustnesscited by 180
  21. 2017Towards Robust Neural Networks via Random Self-ensemblecited by 159
  22. 2018Scaling provable adversarial defensescited by 143
  23. 2018Curriculum Adversarial Trainingcited by 89
  24. 2020Smooth Adversarial Trainingcited by 89
  25. 2019When NAS Meets Robustness: In Search of Robust Architectures against Adversarial Attackscited by 87
  26. 2020Adversarial Distributional Training for Robust Deep Learningcited by 79
  27. 2019Adversarially Robust Generalization Just Requires More Unlabeled Datacited by 76
  28. 2019Improving Adversarial Robustness of Ensembles with Diversity Trainingcited by 69
  29. 2020Weight-Sharing Neural Architecture Search: A Battle to Shrink the Optimization Gapcited by 50
  30. 2019Understanding Architectures Learnt by Cell-based Neural Architecture Searchcited by 48
  31. 2020Once-for-All Adversarial Training: In-Situ Tradeoff between Robustness and Accuracy for Freecited by 48
  32. 2020Single-step Adversarial training with Dropout Schedulingcited by 41
  33. 2019Defending against Whitebox Adversarial Attacks via Randomized Discretizationcited by 29
  34. 2020DSRNA: Differentiable Search of Robust Neural Architecturescited by 27
  35. 2020Adversarially Robust Neural Architecturescited by 22
  36. 2018L2-Nonexpansive Neural Networkscited by 21
  37. 2017Intriguing Properties of Adversarial Examplescited by 20
  38. 2018ResNets Ensemble via the Feynman-Kac Formalism to Improve Natural and Robust Accuraciescited by 18

50 most cited of 62 All 62 (JSON)

Built on by

Cited by year

Citing papers per year: 2021: 6; 2022: 2; 2023: 1; 2024: 2; 2025: 1.

12 papers.

  1. 2022ADBench: Anomaly Detection Benchmarkcited by 232
  2. 2021Better Safe Than Sorry: Preventing Delusive Adversaries with Adversarial Trainingcited by 59
  3. 2021Probabilistic Margins for Instance Reweighting in Adversarial Trainingcited by 57
  4. 2021Reliable Adversarial Distillation with Unreliable Teacherscited by 56
  5. 2021Sparse and Imperceptible Adversarial Attack via a Homotopy Algorithmcited by 15
  6. 2023Defenses in Adversarial Machine Learning: A Surveycited by 14
  7. 2021NoiLIn: Improving Adversarial Training and Correcting Stereotype of Noisy Labelscited by 8
  8. 2022Robust Meta-learning with Sampling Noise and Label Noise via Eigen-Reptilecited by 6
  9. 2024Out-of-Distribution Learning with Human Feedbackcited by 6
  10. 2021Local Reweighting for Adversarial Trainingcited by 1
  11. 2024Investigating and unmasking feature-level vulnerabilities of CNNs to adversarial perturbationscited by 1
  12. 2025ProARD: progressive adversarial robustness distillation: provide wide range of robust studentscited by 0