# Enhancing Adversarial Defense by k-Winners-Take-All

arXiv: https://arxiv.org/abs/1905.10510 · HTML: https://mlsanity.com/paper/1905.10510/

## Key facts

- Cited by 51 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 May 2019.

## Built on by (most cited first, 30 of 51)

- [LLM Lies: Hallucinations are not Bugs, but Features as Adversarial Examples](https://mlsanity.com/paper/2310.01469/) (2023)
- [Adversarial purification with Score-based generative models](https://mlsanity.com/paper/2106.06041/) (2021)
- [Advances in adversarial attacks and defenses in computer vision: A survey](https://mlsanity.com/paper/2108.00401/) (2021)
- [Guided Diffusion Model for Adversarial Purification](https://mlsanity.com/paper/2205.14969/) (2022)
- [Concept Bottleneck Model with Additional Unsupervised Concepts](https://mlsanity.com/paper/2202.01459/) (2022)
- [Meta Gradient Adversarial Attack](https://mlsanity.com/paper/2108.04204/) (2021)
- [Mind the box: l1-APGD for sparse adversarial attacks on image classifiers](https://mlsanity.com/paper/2103.01208/) (2021)
- [Adversarial Attacks are Reversible with Natural Supervision](https://mlsanity.com/paper/2103.14222/) (2021)
- [DISCO: Adversarial Defense with Local Implicit Functions](https://mlsanity.com/paper/2212.05630/) (2022)
- [Automated Discovery of Adaptive Attacks on Adversarial Defenses](https://mlsanity.com/paper/2102.11860/) (2021)
- [CIFS: Improving Adversarial Robustness of CNNs via Channel-wise Importance-based Feature Selection](https://mlsanity.com/paper/2102.05311/) (2021)
- [On the Limitations of Stochastic Pre-processing Defenses](https://mlsanity.com/paper/2206.09491/) (2022)
- [Stereoscopic Universal Perturbations across Different Architectures and Datasets](https://mlsanity.com/paper/2112.06116/) (2021)
- [Sparse Coding in a Dual Memory System for Lifelong Learning](https://mlsanity.com/paper/2301.05058/) (2022)
- [Feature Separation and Recalibration for Adversarial Robustness](https://mlsanity.com/paper/2303.13846/) (2023)
- [AutoAdvExBench: Benchmarking autonomous exploitation of adversarial example defenses](https://mlsanity.com/paper/2503.01811/) (2025)
- [AttackBench: Evaluating Gradient-based Attacks for Adversarial Examples](https://mlsanity.com/paper/2404.19460/) (2024)
- [Increasing Confidence in Adversarial Robustness Evaluations](https://mlsanity.com/paper/2206.13991/) (2022)
- [On the Impact of Hard Adversarial Instances on Overfitting in Adversarial Training](https://mlsanity.com/paper/2112.07324/) (2021)
- [AID-Purifier: A Light Auxiliary Network for Boosting Adversarial Defense](https://mlsanity.com/paper/2107.06456/) (2021)
- [Adaptive Perturbation for Adversarial Attack](https://mlsanity.com/paper/2111.13841/) (2021)
- [Synaptic Plasticity Models and Bio-Inspired Unsupervised Deep Learning: A Survey](https://mlsanity.com/paper/2307.16236/) (2023)
- [The art of defense: letting networks fool the attacker](https://mlsanity.com/paper/2104.02963/) (2021)
- [Robust NAS under adversarial training: benchmark, theory, and beyond](https://mlsanity.com/paper/2403.13134/) (2024)
- [DiffBreak: Is Diffusion-Based Purification Robust?](https://mlsanity.com/paper/2411.16598/) (2024)
- [Robust Binary Models by Pruning Randomly-initialized Networks](https://mlsanity.com/paper/2202.01341/) (2022)
- [On the Convergence of Certified Robust Training with Interval Bound Propagation](https://mlsanity.com/paper/2203.08961/) (2022)
- [Randomness in ML Defenses Helps Persistent Attackers and Hinders Evaluators](https://mlsanity.com/paper/2302.13464/) (2023)
- [ETA: Energy-based Test-time Adaptation for Depth Completion](https://mlsanity.com/paper/2508.05989/) (2025)
- [Interpolation for Robust Learning: Data Augmentation on Wasserstein Geodesics](https://mlsanity.com/paper/2302.02092/) (2023)

## Abstract (arXiv)

We propose a simple change to existing neural network structures for better defending against gradient-based adversarial attacks. Instead of using popular activation functions (such as ReLU), we advocate the use of k-Winners-Take-All (k-WTA) activation, a C0 discontinuous function that purposely invalidates the neural network model's gradient at densely distributed input data points. The proposed k-WTA activation can be readily used in nearly all existing networks and training methods with no significant overhead. Our proposal is theoretically rationalized. We analyze why the discontinuities in k-WTA networks can largely prevent gradient-based search of adversarial examples and why they at the same time remain innocuous to the network training. This understanding is also empirically backed. We test k-WTA activation on various network structures optimized by a training method, be it adversarial training or not. In all cases, the robustness of k-WTA networks outperforms that of traditional networks under white-box attacks.