# Local Competition and Stochasticity for Adversarial Robustness in Deep Learning

arXiv: https://arxiv.org/abs/2101.01121 · HTML: https://mlsanity.com/paper/2101.01121/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 47 references that are in the map.
- First posted on arXiv 4 Jan 2021.

## Builds on (most cited first, 24 of 24)

- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Adversarial examples in the physical world](https://mlsanity.com/paper/1607.02533/) (2016)
- [DeepFool: a simple and accurate method to fool deep neural networks](https://mlsanity.com/paper/1511.04599/) (2015)
- [The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://mlsanity.com/paper/1611.00712/) (2016)
- [Practical Black-Box Attacks against Machine Learning](https://mlsanity.com/paper/1602.02697/) (2016)
- [Ensemble Adversarial Training: Attacks and Defenses](https://mlsanity.com/paper/1705.07204/) (2017)
- [Countering Adversarial Images using Input Transformations](https://mlsanity.com/paper/1711.00117/) (2017)
- [Mitigating Adversarial Effects Through Randomization](https://mlsanity.com/paper/1711.01991/) (2017)
- [Learning from Simulated and Unsupervised Images through Adversarial Training](https://mlsanity.com/paper/1612.07828/) (2016)
- [PixelDefend: Leveraging Generative Models to Understand and Defend against Adversarial Examples](https://mlsanity.com/paper/1710.10766/) (2017)
- [DeepDriving: Learning Affordance for Direct Perception in Autonomous Driving](https://mlsanity.com/paper/1505.00256/) (2015)
- [Stochastic Activation Pruning for Robust Adversarial Defense](https://mlsanity.com/paper/1803.01442/) (2018)
- [Deflecting Adversarial Attacks with Pixel Deflection](https://mlsanity.com/paper/1801.08926/) (2018)
- [The Limitations of Adversarial Training and the Blind-Spot Attack](https://mlsanity.com/paper/1901.04684/) (2019)
- [Black-box Adversarial Attacks on Video Recognition Models](https://mlsanity.com/paper/1904.05181/) (2019)
- [APE-GAN: Adversarial Perturbation Elimination with GAN](https://mlsanity.com/paper/1707.05474/) (2017)
- [The Robust Manifold Defense: Adversarial Training using Generative Models](https://mlsanity.com/paper/1712.09196/) (2017)
- [Hold me tight! Influence of discriminative features on deep network boundaries](https://mlsanity.com/paper/2002.06349/) (2020)
- [Optimism in the Face of Adversity: Understanding and Improving Deep Learning through Adversarial Robustness](https://mlsanity.com/paper/2010.09624/) (2020)
- [Nonparametric Bayesian Deep Networks with Local Competition](https://mlsanity.com/paper/1805.07624/) (2018)
- [Simple Physical Adversarial Examples against End-to-End Autonomous Driving Models](https://mlsanity.com/paper/1903.05157/) (2019)
- [VectorDefense: Vectorization as a Defense to Adversarial Examples](https://mlsanity.com/paper/1804.08529/) (2018)

## Built on by (most cited first, 12 of 12)

- [Rethinking Bayesian Learning for Data Analysis: The Art of Prior and Inference in Sparsity-Aware Modeling](https://mlsanity.com/paper/2205.14283/) (2022)
- [Stochastic Local Winner-Takes-All Networks Enable Profound Adversarial Robustness](https://mlsanity.com/paper/2112.02671/) (2021)
- [DISCOVER: Making Vision Networks Interpretable via Competition and Dissection](https://mlsanity.com/paper/2310.04929/) (2023)
- [Stochastic Deep Networks with Linear Competing Units for Model-Agnostic Meta-Learning](https://mlsanity.com/paper/2208.01573/) (2022)
- [Synaptic Plasticity Models and Bio-Inspired Unsupervised Deep Learning: A Survey](https://mlsanity.com/paper/2307.16236/) (2023)
- [A New Dataset for End-to-End Sign Language Translation: The Greek Elementary School Dataset](https://mlsanity.com/paper/2310.04753/) (2023)
- [Competing Mutual Information Constraints with Stochastic Competition-based Activations for Learning Diversified Representations](https://mlsanity.com/paper/2201.03624/) (2022)
- [On the Role of Randomization in Adversarially Robust Classification](https://mlsanity.com/paper/2302.07221/) (2023)
- [Continual Deep Learning on the Edge via Stochastic Local Competition among Subnetworks](https://mlsanity.com/paper/2407.10758/) (2024)
- [Improving the Behaviour of Vision Transformers with Token-consistent Stochastic Layers](https://mlsanity.com/paper/2112.15111/) (2021)
- [Unveiling the Role of Randomization in Multiclass Adversarial Classification: Insights from Graph Theory](https://mlsanity.com/paper/2503.14299/) (2025)
- [Colliding with Adversaries at ECML-PKDD 2025 Model Robustness Competition 1st Prize Solution](https://mlsanity.com/paper/2510.16443/) (2025)

## Abstract (arXiv)

This work addresses adversarial robustness in deep learning by considering deep networks with stochastic local winner-takes-all (LWTA) activations. This type of network units result in sparse representations from each model layer, as the units are organized in blocks where only one unit generates a non-zero output. The main operating principle of the introduced units lies on stochastic arguments, as the network performs posterior sampling over competing units to select the winner. We combine these LWTA arguments with tools from the field of Bayesian non-parametrics, specifically the stick-breaking construction of the Indian Buffet Process, to allow for inferring the sub-part of each layer that is essential for modeling the data at hand. Then, inference is performed by means of stochastic variational Bayes. We perform a thorough experimental evaluation of our model using benchmark datasets. As we show, our method achieves high robustness to adversarial perturbations, with state-of-the-art performance in powerful adversarial attack schemes.