# The Feasibility and Inevitability of Stealth Attacks

arXiv: https://arxiv.org/abs/2106.13997 · HTML: https://mlsanity.com/paper/2106.13997/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 45 references that are in the map.
- First posted on arXiv 26 Jun 2021.

## Builds on (most cited first, 21 of 21)

- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples](https://mlsanity.com/paper/1802.00420/) (2018)
- [BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain](https://mlsanity.com/paper/1708.06733/) (2017)
- [What is the State of Neural Network Pruning?](https://mlsanity.com/paper/2003.03033/) (2020)
- [One pixel attack for fooling deep neural networks](https://mlsanity.com/paper/1710.08864/) (2017)
- [Pruning neural networks without any data by iteratively conserving synaptic flow](https://mlsanity.com/paper/2006.05467/) (2020)
- [Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey](https://mlsanity.com/paper/1801.00553/) (2018)
- [Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning](https://mlsanity.com/paper/1712.03141/) (2017)
- [Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks](https://mlsanity.com/paper/2103.14749/) (2021)
- [What is being transferred in transfer learning?](https://mlsanity.com/paper/2008.11687/) (2020)
- [A Survey of Model Compression and Acceleration for Deep Neural Networks](https://mlsanity.com/paper/1710.09282/) (2017)
- [On instabilities of deep learning in image reconstruction - Does AI come at a cost?](https://mlsanity.com/paper/1902.05300/) (2019)
- [Are adversarial examples inevitable?](https://mlsanity.com/paper/1809.02104/) (2018)
- [Universal Adversarial Training](https://mlsanity.com/paper/1811.11304/) (2018)
- [Can stable and accurate neural networks be computed? -- On the barriers of deep learning and Smale's 18th problem](https://mlsanity.com/paper/2101.08286/) (2021)
- [SafetyNets: Verifiable Execution of Deep Neural Networks on an Untrusted Cloud](https://mlsanity.com/paper/1706.10268/) (2017)
- [A Game-Based Approximate Verification of Deep Neural Networks with Provable Guarantees](https://mlsanity.com/paper/1807.03571/) (2018)
- [Correction of AI systems by linear discriminants: Probabilistic foundations](https://mlsanity.com/paper/1811.05321/) (2018)
- [On Adversarial Examples and Stealth Attacks in Artificial Intelligence Systems](https://mlsanity.com/paper/2004.04479/) (2020)
- [The unreasonable effectiveness of small neural ensembles in high-dimensional brain](https://mlsanity.com/paper/1809.07656/) (2018)
- [Mitigating Advanced Adversarial Attacks with More Advanced Gradient Obfuscation Techniques](https://mlsanity.com/paper/2005.13712/) (2020)

## Built on by (most cited first, 12 of 12)

- [The mathematics of adversarial attacks in AI -- Why deep learning is unstable despite the existence of stable neural networks](https://mlsanity.com/paper/2109.06098/) (2021)
- [High-dimensional separability for one- and few-shot learning](https://mlsanity.com/paper/2106.15416/) (2021)
- [Adversarial Ink: Componentwise Backward Error Attacks on Deep Learning](https://mlsanity.com/paper/2306.02918/) (2023)
- [Adversarial Parameter Attack on Deep Neural Networks](https://mlsanity.com/paper/2203.10502/) (2022)
- [The Boundaries of Verifiable Accuracy, Robustness, and Generalisation in Deep Learning](https://mlsanity.com/paper/2309.07072/) (2023)
- [Stealth edits to large language models](https://mlsanity.com/paper/2406.12670/) (2024)
- [Vulnerability Analysis of Transformer-based Optical Character Recognition to Adversarial Attacks](https://mlsanity.com/paper/2311.17128/) (2023)
- [Do stable neural networks exist for classification problems? -- A new view on stability in AI](https://mlsanity.com/paper/2401.07874/) (2024)
- [Deceptive Diffusion: Generating Synthetic Adversarial Examples](https://mlsanity.com/paper/2406.19807/) (2024)
- [Staining and locking computer vision models without retraining](https://mlsanity.com/paper/2507.22000/) (2025)
- [Can We Rely on AI?](https://mlsanity.com/paper/2308.15092/) (2023)
- [Updating the standard neuron model in artificial neural networks](https://mlsanity.com/paper/2605.30370/) (2026)

## Abstract (arXiv)

We develop and study new adversarial perturbations that enable an attacker to gain control over decisions in generic Artificial Intelligence (AI) systems including deep learning neural networks. In contrast to adversarial data modification, the attack mechanism we consider here involves alterations to the AI system itself. Such a stealth attack could be conducted by a mischievous, corrupt or disgruntled member of a software development team. It could also be made by those wishing to exploit a ``democratization of AI'' agenda, where network architectures and trained parameter sets are shared publicly. We develop a range of new implementable attack strategies with accompanying analysis, showing that with high probability a stealth attack can be made transparent, in the sense that system performance is unchanged on a fixed validation set which is unknown to the attacker, while evoking any desired output on a trigger input of interest. The attacker only needs to have estimates of the size of the validation set and the spread of the AI's relevant latent space. In the case of deep learning neural networks, we show that a one neuron attack is possible - a modification to the weights and bias associated with a single neuron - revealing a vulnerability arising from over-parameterization. We illustrate these concepts using state of the art architectures on two standard image data sets. Guided by the theory and computational results, we also propose strategies to guard against stealth attacks.