mlsanityData to 7 Oct 2026

Improving Adversarial Robustness via Channel-wise Activation Suppressing

arXiv 2103.08307 ↗First posted 11 Mar 2021cs.LGcited by 76

Not on the map. Why?

Abstract (arXiv)

The study of adversarial examples and their activation has attracted significant attention for secure and robust learning with deep neural networks (DNNs). Different from existing works, in this paper, we highlight two new characteristics of adversarial examples from the channel-wise activation perspective: 1) the activation magnitudes of adversarial examples are higher than that of natural examples; and 2) the channels are activated more uniformly by adversarial examples than natural examples. We find that the state-of-the-art defense adversarial training has addressed the first issue of high activation magnitudes via training on adversarial examples, while the second issue of uniform activation remains. This motivates us to suppress redundant activation from being activated by adversarial perturbations via a Channel-wise Activation Suppressing (CAS) strategy. We show that CAS can train a model that inherently suppresses adversarial activation, and can be easily applied to existing defense methods to further improve their robustness. Our work provides a simple but generic training strategy for robustifying the intermediate layer activation of DNNs.

Key facts. Cited by 76 papers in the map (data through 7 Oct 2026). Builds on 38 of its 53 references that are in the map. First posted on arXiv 11 Mar 2021.

Builds on

38 of 53 references.

References · 38

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  4. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  5. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  6. 2013Intriguing properties of neural networkscited by 5,114
  7. 2016Wide Residual Networkscited by 3,057
  8. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
  9. 2016Adversarial examples in the physical worldcited by 1,762
  10. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
  11. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,442
  12. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,259
26 more
  1. 2019Adversarial Examples Are Not Bugs, They Are Featurescited by 950
  2. 2015Distillation as a Defense to Adversarial Perturbations against Deep Neural Networkscited by 675
  3. 2020On Adaptive Attacks to Adversarial Example Defensescited by 547
  4. 2016Delving into Transferable Adversarial Examples and Black-box Attackscited by 530
  5. 2020Overfitting in adversarially robust deep learningcited by 504
  6. 2018Efficient Neural Network Robustness Certification with General Activation Functionscited by 439
  7. 2019Minimally distorted Adversarial Examples with a Fast Adaptive Boundary Attackcited by 333
  8. 2017Defense against Adversarial Attacks Using High-Level Representation Guided Denoisercited by 329
  9. 2018Feature Denoising for Improving Adversarial Robustnesscited by 312
  10. 2018Characterizing Adversarial Subspaces Using Local Intrinsic Dimensionalitycited by 280
  11. 2018Stochastic Activation Pruning for Robust Adversarial Defensecited by 171
  12. 2020Skip Connections Matter: On the Transferability of Adversarial Examples Generated with ResNetscited by 165
  13. 2019Understanding Adversarial Attacks on Deep Learning Based Medical Image Analysis Systemscited by 151
  14. 2014Towards Deep Neural Network Architectures Robust to Adversarial Examplescited by 146
  15. 2020Adversarial Camouflage: Hiding Physical-World Attacks with Natural Stylescited by 126
  16. 2019NATTACK: Learning the Distributions of Adversarial Examples for an Improved Black-Box Attack on Deep Neural Networkscited by 94
  17. 2020A Unified Approach to Interpreting and Boosting Adversarial Transferabilitycited by 71
  18. 2019Adversarial Defense by Restricting the Hidden Space of Deep Neural Networkscited by 56
  19. 2020Once-for-All Adversarial Training: In-Situ Tradeoff between Robustness and Accuracy for Freecited by 48
  20. 2019Batch Normalization is a Cause of Adversarial Vulnerabilitycited by 28
  21. 2019Harnessing the Vulnerability of Latent Layers in Adversarially Trained Modelscited by 26
  22. 2017Residual Convolutional CTC Networks for Automatic Speech Recognitioncited by 21
  23. 2018Defend Deep Neural Networks Against Adversarial Examples via Fixed and Dynamic Quantized Activation Functionscited by 17
  24. 2019Interpreting Adversarial Examples by Activation Promotion and Suppressioncited by 17
  25. 2018Improved robustness to adversarial examples using Lipschitz regularization of the losscited by 13
  26. 2018Adversarial Defense via Data Dependent Activation Function and Total Variation Minimizationcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 19; 2022: 16; 2023: 17; 2024: 14; 2025: 9; 2026: 1.

76 papers.

Sort:

By area:

  1. 2022LAS-AT: Adversarial Training with Learnable Attack Strategycited by 94
  2. 2021Revisiting Adversarial Robustness Distillation: Robust Soft Labels Make Student Bettercited by 77
  3. 2021Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networkscited by 73
  4. 2024Inducing High Energy-Latency of Large Vision-Language Models with Verbose Imagescited by 69
  5. 2021Reliable Adversarial Distillation with Unreliable Teacherscited by 55
  6. 2022Robust Semantic Communications with Masked VQ-VAE Enabled Codebookcited by 53
  7. 2022Understanding Robust Overfitting of Adversarial Training and Beyondcited by 50
  8. 2023Robust Principles: Architectural Design Principles for Adversarially Robust CNNscited by 46
  9. 2023Backdoor Defense via Adaptively Splitting Poisoned Datasetcited by 43
  10. 2024A Theoretical Understanding of Self-Correction through In-context Alignmentcited by 43
  11. 2023Towards Compact 3D Representations via Point Feature Enhancement Masked Autoencoderscited by 42
  12. 2021Boosting Fast Adversarial Training with Learnable Adversarial Initializationcited by 40
38 more
  1. 2022Self-Ensemble Adversarial Training for Improved Robustnesscited by 38
  2. 2022A Unified Wasserstein Distributional Robustness Framework for Adversarial Trainingcited by 36
  3. 2022Prior-Guided Adversarial Initialization for Fast Adversarial Trainingcited by 34
  4. 2023Improving Fast Adversarial Training with Prior-Guided Knowledgecited by 34
  5. 2021Analysis and Applications of Class-wise Robustness in Adversarial Trainingcited by 33
  6. 2021Towards Robust Neural Image Compression: Adversarial Attack and Model Finetuningcited by 24
  7. 2021On the Effect of Pruning on Adversarial Robustnesscited by 23
  8. 2024Adversarial Robustness for Visual Grounding of Multimodal Large Language Modelscited by 23
  9. 2023GMMFormer: Gaussian-Mixture-Model Based Transformer for Efficient Partially Relevant Video Retrievalcited by 20
  10. 2021Removing Adversarial Noise in Class Activation Feature Spacecited by 18
  11. 2023Balance, Imbalance, and Rebalance: Understanding Robust Overfitting from a Minimax Game Perspectivecited by 18
  12. 2021A Unified Game-Theoretic Interpretation of Adversarial Robustnesscited by 17
  13. 2023Feature Separation and Recalibration for Adversarial Robustnesscited by 16
  14. 2023Adversarial Examples Are Not Real Featurescited by 16
  15. 2022On the Effectiveness of Adversarial Training against Backdoor Attackscited by 15
  16. 2024DAT: Improving Adversarial Robustness via Generative Amplitude Mix-up in Frequency Domaincited by 12
  17. 2021AGKD-BML: Defense Against Adversarial Attack by Attention Guided Knowledge Distillation and Bi-directional Metric Learningcited by 10
  18. 2023TranSegPGD: Improving Transferability of Adversarial Examples on Semantic Segmentationcited by 10
  19. 2021MixACM: Mixup-Based Robustness Transfer via Distillation of Activated Channel Mapscited by 9
  20. 2022Adversarial Robustness through the Lens of Convolutional Filterscited by 9
  21. 2022A.I. Robustness: a Human-Centered Perspective on Technological Challenges and Opportunitiescited by 8
  22. 2023Fast Propagation is Better: Accelerating Single-Step Adversarial Training via Sampling Subnetworkscited by 8
  23. 2021Instance Correction for Learning with Open-set Noisy Labelscited by 7
  24. 2024Video Watermarking: Safeguarding Your Video from (Unauthorized) Annotations by Video-based LLMscited by 7
  25. 2024AdvQDet: Detecting Query-Based Adversarial Attacks with Adversarial Contrastive Prompt Tuningcited by 7
  26. 2023RobArch: Designing Robust Architectures against Adversarial Attackscited by 6
  27. 2022Towards an Awareness of Time Series Anomaly Detection Models' Adversarial Vulnerabilitycited by 5
  28. 2023Indirect Gradient Matching for Adversarial Robust Distillationcited by 5
  29. 2025Long-tailed Adversarial Training with Self-Distillationcited by 5
  30. 2021Clustering Effect of (Linearized) Adversarial Robust Modelscited by 3
  31. 2023Benchmarking Adversarial Robustness of Compressed Deep Learning Modelscited by 3
  32. 2024Data-Driven Lipschitz Continuity: A Cost-Effective Approach to Improve Adversarial Robustnesscited by 3
  33. 2024Interpretability-Guided Test-Time Adversarial Defensecited by 3
  34. 2024Adversarial Attacks on Hyperbolic Networkscited by 3
  35. 2024CausAdv: A Causal-based Framework for Detecting Adversarial Examplescited by 2
  36. 2025Test-Time Defense Against Adversarial Attacks via Stochastic Resonance of Latent Ensemblescited by 2
  37. 2021Dual Head Adversarial Trainingcited by 1
  38. 2021Less is More: Feature Selection for Adversarial Robustness with Compressive Counter-Adversarial Attackscited by 1

50 most cited of 76 All 76 (JSON)