mlsanityData to 7 Oct 2026

Enhancing Adversarial Defense by k-Winners-Take-All

arXiv 1905.10510 ↗First posted 25 May 2019cs.LG cs.AI cs.CRcited by 51

Not on the map. Why?

Abstract (arXiv)

We propose a simple change to existing neural network structures for better defending against gradient-based adversarial attacks. Instead of using popular activation functions (such as ReLU), we advocate the use of k-Winners-Take-All (k-WTA) activation, a C0 discontinuous function that purposely invalidates the neural network model's gradient at densely distributed input data points. The proposed k-WTA activation can be readily used in nearly all existing networks and training methods with no significant overhead. Our proposal is theoretically rationalized. We analyze why the discontinuities in k-WTA networks can largely prevent gradient-based search of adversarial examples and why they at the same time remain innocuous to the network training. This understanding is also empirically backed. We test k-WTA activation on various network structures optimized by a training method, be it adversarial training or not. In all cases, the robustness of k-WTA networks outperforms that of traditional networks under white-box attacks.

Key facts. Cited by 51 papers in the map (data through 7 Oct 2026). First posted on arXiv 25 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 18; 2022: 12; 2023: 8; 2024: 5; 2025: 6; 2026: 2.

51 papers.

Sort:

By area:

  1. 2023LLM Lies: Hallucinations are not Bugs, but Features as Adversarial Examplescited by 177
  2. 2021Adversarial purification with Score-based generative modelscited by 132
  3. 2021Advances in adversarial attacks and defenses in computer vision: A surveycited by 127
  4. 2022Guided Diffusion Model for Adversarial Purificationcited by 90
  5. 2022Concept Bottleneck Model with Additional Unsupervised Conceptscited by 79
  6. 2021Meta Gradient Adversarial Attackcited by 55
  7. 2021Mind the box: l1-APGD for sparse adversarial attacks on image classifierscited by 50
  8. 2021Adversarial Attacks are Reversible with Natural Supervisioncited by 47
  9. 2022DISCO: Adversarial Defense with Local Implicit Functionscited by 33
  10. 2021Automated Discovery of Adaptive Attacks on Adversarial Defensescited by 22
  11. 2021CIFS: Improving Adversarial Robustness of CNNs via Channel-wise Importance-based Feature Selectioncited by 21
  12. 2022On the Limitations of Stochastic Pre-processing Defensescited by 21
38 more
  1. 2021Stereoscopic Universal Perturbations across Different Architectures and Datasetscited by 17
  2. 2022Sparse Coding in a Dual Memory System for Lifelong Learningcited by 16
  3. 2023Feature Separation and Recalibration for Adversarial Robustnesscited by 16
  4. 2024AttackBench: Evaluating Gradient-based Attacks for Adversarial Examplescited by 14
  5. 2025AutoAdvExBench: Benchmarking autonomous exploitation of adversarial example defensescited by 14
  6. 2022Increasing Confidence in Adversarial Robustness Evaluationscited by 12
  7. 2021On the Impact of Hard Adversarial Instances on Overfitting in Adversarial Trainingcited by 11
  8. 2021AID-Purifier: A Light Auxiliary Network for Boosting Adversarial Defensecited by 10
  9. 2023Synaptic Plasticity Models and Bio-Inspired Unsupervised Deep Learning: A Surveycited by 7
  10. 2021The art of defense: letting networks fool the attackercited by 6
  11. 2021Adaptive Perturbation for Adversarial Attackcited by 6
  12. 2024Robust NAS under adversarial training: benchmark, theory, and beyondcited by 6
  13. 2024DiffBreak: Is Diffusion-Based Purification Robust?cited by 6
  14. 2022Robust Binary Models by Pruning Randomly-initialized Networkscited by 5
  15. 2022On the Convergence of Certified Robust Training with Interval Bound Propagationcited by 5
  16. 2023Randomness in ML Defenses Helps Persistent Attackers and Hinders Evaluatorscited by 5
  17. 2025ETA: Energy-based Test-time Adaptation for Depth Completioncited by 4
  18. 2023Interpolation for Robust Learning: Data Augmentation on Wasserstein Geodesicscited by 3
  19. 2024Interpretability-Guided Test-Time Adversarial Defensecited by 3
  20. 2022LADDER: Latent Boundary-guided Adversarial Trainingcited by 2
  21. 2022Attacking Adversarial Defences by Smoothing the Loss Landscapecited by 2
  22. 2025Test-Time Defense Against Adversarial Attacks via Stochastic Resonance of Latent Ensemblescited by 2
  23. 2021Improving Robustness for Pose Estimation via Stable Heatmap Regressioncited by 1
  24. 2021Less is More: Feature Selection for Adversarial Robustness with Compressive Counter-Adversarial Attackscited by 1
  25. 2021Simple black-box universal adversarial attacks on medical image classification based on deep neural networkscited by 1
  26. 2021Denoised Internal Models: a Brain-Inspired Autoencoder against Adversarial Attackscited by 1
  27. 2022Efficient and Robust Classification for Sparse Attackscited by 1
  28. 2023IRAD: Implicit Representation-driven Image Resampling against Adversarial Attackscited by 1
  29. 2025On the Feasibility of Poisoning Text-to-Image AI Models via Adversarial Mislabelingcited by 1
  30. 2021Spatio-Temporal Sparsification for General Robust Graph Convolution Networkscited by 0
  31. 2021Introducing the DOME Activation Functionscited by 0
  32. 2022Symmetry Defense Against CNN Adversarial Perturbation Attackscited by 0
  33. 2023Revisiting and Advancing Adversarial Training Through A Simple Baselinecited by 0
  34. 2023Enhancing Robust Representation in Adversarial Training: Alignment and Exclusion Criteriacited by 0
  35. 2024Enhance DNN Adversarial Robustness and Efficiency via Injecting Noise to Non-Essential Neuronscited by 0
  36. 2025Instant Video Models: Universal Adapters for Stabilizing Image-Based Networkscited by 0
  37. 2025Studying Various Activation Functions and Non-IID Data for Machine Learning Model Robustnesscited by 0
  38. 2026Improving Adversarial Robustness via Activation Amplification and Attenuationcited by 0

50 most cited of 51 All 51 (JSON)