mlsanityData to 14 Oct 2026

Conserve-Update-Revise to Cure Generalization and Robustness Trade-off in Adversarial Training

arXiv 2401.14948 ↗First posted 26 Jan 2024cs.LG cs.AI cs.CVcited by 6

Not on the map. Why?

Abstract (arXiv)

Adversarial training improves the robustness of neural networks against adversarial attacks, albeit at the expense of the trade-off between standard and robust generalization. To unveil the underlying factors driving this phenomenon, we examine the layer-wise learning capabilities of neural networks during the transition from a standard to an adversarial setting. Our empirical findings demonstrate that selectively updating specific layers while preserving others can substantially enhance the network's learning capacity. We therefore propose CURE, a novel training framework that leverages a gradient prominence criterion to perform selective conservation, updating, and revision of weights. Importantly, CURE is designed to be dataset- and architecture-agnostic, ensuring its applicability across various scenarios. It effectively tackles both memorization and overfitting issues, thus enhancing the trade-off between robustness and generalization and additionally, this training approach also aids in mitigating "robust overfitting". Furthermore, our study provides valuable insights into the mechanisms of selective adversarial training and offers a promising avenue for future research.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 42 of its 49 references that are in the map. First posted on arXiv 26 Jan 2024.

Builds on

42 of 49 references.

References · 42

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  3. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  4. 2013Intriguing properties of neural networkscited by 5,155
  5. 2016Wide Residual Networkscited by 3,070
  6. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,998
  7. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,877
  8. 2016Identity Mappings in Deep Residual Networkscited by 1,965
  9. 2019Similarity of Neural Network Representations Revisitedcited by 1,755
  10. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,517
  11. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,451
  12. 2019Adversarial Examples Are Not Bugs, They Are Featurescited by 953
30 more
  1. 2018Robustness May Be at Odds with Accuracycited by 880
  2. 2017Ensemble Adversarial Training: Attacks and Defensescited by 840
  3. 2019Square Attack: a query-efficient black-box adversarial attack via random searchcited by 685
  4. 2020Overfitting in adversarially robust deep learningcited by 506
  5. 2019Unlabeled Data Improves Adversarial Robustnesscited by 416
  6. 2019Minimally distorted Adversarial Examples with a Fast Adaptive Boundary Attackcited by 334
  7. 2018Adversarially Robust Generalization Requires More Datacited by 318
  8. 2020Attacks Which Do Not Kill Training Make Adversarial Learning Strongercited by 243
  9. 2020Geometry-aware Instance-reweighted Adversarial Trainingcited by 198
  10. 2019Adversarial Training Can Hurt Generalizationcited by 132
  11. 2019Adversarially Robust Distillationcited by 125
  12. 2020Training BatchNorm and Only BatchNorm: On the Expressive Power of Random Features in CNNscited by 104
  13. 2022LAS-AT: Adversarial Training with Learnable Attack Strategycited by 95
  14. 2021Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networkscited by 74
  15. 2020Adversarial Vertex Mixup: Toward Better Adversarially Robust Generalizationcited by 73
  16. 2021Reliable Adversarial Distillation with Unreliable Teacherscited by 56
  17. 2021Exploring Memorization in Adversarial Trainingcited by 55
  18. 2019Robustness to Adversarial Perturbations in Learning from Incomplete Datacited by 49
  19. 2023Data Augmentation Alone Can Improve Adversarial Trainingcited by 43
  20. 2019Adversarial Robustness May Be at Odds With Simplicitycited by 39
  21. 2022Self-Ensemble Adversarial Training for Improved Robustnesscited by 38
  22. 2020Efficient Robust Training via Backward Smoothingcited by 29
  23. 2020RIFLE: Backpropagation in Depth for Deep Transfer Learning through Re-Initializing the Fully-connected LayErcited by 27
  24. 2020SAT: Improving Adversarial Training via Curriculum-Based Loss Smoothingcited by 26
  25. 2021The Impact of Reinitialization on Generalization in Convolutional Neural Networkscited by 20
  26. 2021Mutual Adversarial Training: Learning together is better than going alonecited by 17
  27. 2020Adversarial Training with Stochastic Weight Averagecited by 12
  28. 2022Squeeze Training for Adversarial Robustnesscited by 10
  29. 2020Adversarial Concurrent Training: Optimizing Robustness and Accuracy Trade-off of Deep Neural Networkscited by 9
  30. 2021Guided Interpolation for Adversarial Trainingcited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 2; 2026: 2.

6 papers.

  1. 2024Passive Deepfake Detection Across Multi-modalities: A Comprehensive Surveycited by 4
  2. 2025Failure Cases Are Better Learned But Boundary Says Sorry: Facilitating Smooth Perception Change for Accuracy-Robustness Trade-Off in Adversarial Trainingcited by 3
  3. 2024New Paradigm of Adversarial Training: Releasing Accuracy-Robustness Trade-Off via Dummy Classcited by 2
  4. 2025Generalist++: A Meta-learning Framework for Mitigating Trade-off in Adversarial Trainingcited by 1
  5. 2026Robust Alignment: Harmonizing Clean Accuracy and Adversarial Robustness in Adversarial Trainingcited by 0
  6. 2026Improving the Robustness/Accuracy Tradeoff Against Adversarial Attacks Using Information Bottleneck Distillation Through Dual Teacherscited by 0