mlsanityData to 7 Oct 2026

Fast AdvProp

arXiv 2204.09838 ↗First posted 21 Apr 2022cs.CVcited by 6

Not on the map. Why?

Abstract (arXiv)

Adversarial Propagation (AdvProp) is an effective way to improve recognition models, leveraging adversarial examples. Nonetheless, AdvProp suffers from the extremely slow training speed, mainly because: a) extra forward and backward passes are required for generating adversarial examples; b) both original samples and their adversarial counterparts are used for training (i.e., 2× data). In this paper, we introduce Fast AdvProp, which aggressively revamps AdvProp's costly training components, rendering the method nearly as cheap as the vanilla training. Specifically, our modifications in Fast AdvProp are guided by the hypothesis that disentangled learning with adversarial examples is the key for performance improvements, while other training recipes (e.g., paired clean and adversarial training samples, multi-step adversarial attackers) could be largely simplified. Our empirical results show that, compared to the vanilla training baseline, Fast AdvProp is able to further model performance on a spectrum of visual benchmarks, without incurring extra training cost. Additionally, our ablations find Fast AdvProp scales better if larger models are used, is compatible with existing data augmentation methods (i.e., Mixup and CutMix), and can be easily adapted to other recognition tasks like object detection. The code is available here: https://github.com/meijieru/fastadvprop.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 44 of its 50 references that are in the map. First posted on arXiv 21 Apr 2022.

Builds on

44 of 50 references.

References · 44

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  5. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  6. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  7. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  8. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  9. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  10. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  11. 2013Intriguing properties of neural networkscited by 5,114
  12. 2014Going Deeper with Convolutionscited by 4,231
32 more
  1. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,875
  2. 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 1,998
  3. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,842
  4. 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,623
  5. 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,430
  6. 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,304
  7. 2016Adversarial Machine Learning at Scalecited by 934
  8. 2017Revisiting Unreasonable Effectiveness of Data in Deep Learning Eracited by 916
  9. 2017Focal Loss for Dense Object Detectioncited by 749
  10. 2018AutoAugment: Learning Augmentation Policies from Datacited by 728
  11. 2020Fast is better than free: Revisiting adversarial trainingcited by 723
  12. 2019Adversarial Training for Free!cited by 638
  13. 2018Exploring the Limits of Weakly Supervised Pretrainingcited by 604
  14. 2018The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scalecited by 542
  15. 2021High-Performance Large-Scale Image Recognition Without Normalizationcited by 359
  16. 2019Benchmarking Robustness in Object Detection: Autonomous Driving when Winter is Comingcited by 332
  17. 2019Adversarial Examples Improve Image Recognitioncited by 283
  18. 2020Understanding and Improving Fast Adversarial Trainingcited by 213
  19. 2020Robust Pre-Training by Adversarial Contrastive Learningcited by 144
  20. 2019Adversarial Training Can Hurt Generalizationcited by 131
  21. 2018Benchmarking Neural Network Robustness to Common Corruptions and Surface Variationscited by 99
  22. 2020Contrastive Learning with Adversarial Examplescited by 91
  23. 2020Shape-Texture Debiased Neural Network Trainingcited by 76
  24. 2019Interpreting Adversarially Trained Convolutional Neural Networkscited by 75
  25. 2020Once-for-All Adversarial Training: In-Situ Tradeoff between Robustness and Accuracy for Freecited by 48
  26. 2021Robust and Accurate Object Detection via Adversarial Learningcited by 42
  27. 2021Pyramid Adversarial Training Improves ViT Performancecited by 39
  28. 2021Adversarial Feature Augmentation and Normalization for Visual Recognitioncited by 11
  29. 2021Concurrent Adversarial Learning for Large-Batch Trainingcited by 11
  30. 2020Adversarial Momentum-Contrastive Pre-Trainingcited by 10
  31. 2020Does Data Augmentation Benefit from Split BatchNormscited by 7
  32. 2021Robust and Accurate Object Detection via Self-Knowledge Distillationcited by 3

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 1; 2024: 3.

6 papers.

  1. 2022Enhance the Visual Representation via Discrete Adversarial Trainingcited by 23
  2. 2023Distilling Out-of-Distribution Robustness from Vision-Language Foundation Modelscited by 9
  3. 2022Dynamic Test-Time Augmentation via Differentiable Functionscited by 4
  4. 2024Test-time Adaptation Meets Image Enhancement: Improving Accuracy via Uncertainty-aware Logit Switchingcited by 2
  5. 2024AEMIM: Adversarial Examples Meet Masked Image Modelingcited by 1
  6. 2024EntProp: High Entropy Propagation for Improving Accuracy and Robustnesscited by 0