mlsanityData to 7 Oct 2026

Stochastic Local Winner-Takes-All Networks Enable Profound Adversarial Robustness

arXiv 2112.02671 ↗First posted 5 Dec 2021cs.LG stat.MLcited by 11

Not on the map. Why?

Abstract (arXiv)

This work explores the potency of stochastic competition-based activations, namely Stochastic Local Winner-Takes-All (LWTA), against powerful (gradient-based) white-box and black-box adversarial attacks; we especially focus on Adversarial Training settings. In our work, we replace the conventional ReLU-based nonlinearities with blocks comprising locally and stochastically competing linear units. The output of each network layer now yields a sparse output, depending on the outcome of winner sampling in each block. We rely on the Variational Bayesian framework for training and inference; we incorporate conventional PGD-based adversarial training arguments to increase the overall adversarial robustness. As we experimentally show, the arising networks yield state-of-the-art robustness against powerful adversarial attacks while retaining very high classification rate in the benign case.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 17 of its 20 references that are in the map. First posted on arXiv 5 Dec 2021.

Builds on

17 of 20 references.

References · 17

  1. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  2. 2013Auto-Encoding Variational Bayescited by 3,980
  3. 2016Wide Residual Networkscited by 3,057
  4. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,442
  5. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,259
  6. 2016The Concrete Distribution: A Continuous Relaxation of Discrete Random Variablescited by 1,135
  7. 2020Overfitting in adversarially robust deep learningcited by 504
  8. 2019Unlabeled Data Improves Adversarial Robustnesscited by 415
  9. 2019Using Pre-Training Can Improve Model Robustness and Uncertaintycited by 373
  10. 2020Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examplescited by 275
  11. 2020Attacks Which Do Not Kill Training Make Adversarial Learning Strongercited by 242
  12. 2020Boosting Adversarial Training with Hypersphere Embeddingcited by 97
5 more
  1. 2020Do Wider Neural Networks Really Help Adversarial Robustness?cited by 79
  2. 2019The Limitations of Adversarial Training and the Blind-Spot Attackcited by 63
  3. 2021Stochastic Transformer Networks with Linear Competing Units: Application to end-to-end SL Translationcited by 35
  4. 2018Nonparametric Bayesian Deep Networks with Local Competitioncited by 17
  5. 2021Local Competition and Stochasticity for Adversarial Robustness in Deep Learningcited by 12

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 4; 2024: 2; 2025: 3.

11 papers.

  1. 2022Rethinking Bayesian Learning for Data Analysis: The Art of Prior and Inference in Sparsity-Aware Modelingcited by 12
  2. 2023DISCOVER: Making Vision Networks Interpretable via Competition and Dissectioncited by 8
  3. 2023Synaptic Plasticity Models and Bio-Inspired Unsupervised Deep Learning: A Surveycited by 7
  4. 2023A New Dataset for End-to-End Sign Language Translation: The Greek Elementary School Datasetcited by 6
  5. 2022Competing Mutual Information Constraints with Stochastic Competition-based Activations for Learning Diversified Representationscited by 4
  6. 2023On the Role of Randomization in Adversarially Robust Classificationcited by 4
  7. 2024Causal Perception Inspired Representation Learning for Trustworthy Image Quality Assessmentcited by 0
  8. 2024Enhancing Adversarial Robustness through Multi-Objective Representation Learningcited by 0
  9. 2025Lattice Climber Attack: Adversarial attacks for randomized mixtures of classifierscited by 0
  10. 2025Neuro-inspired Ensemble-to-Ensemble Communication Primitives for Sparse and Efficient ANNscited by 0
  11. 2025Colliding with Adversaries at ECML-PKDD 2025 Model Robustness Competition 1st Prize Solutioncited by 0