# TnT Attacks! Universal Naturalistic Adversarial Patches Against Deep Neural Network Systems

arXiv: https://arxiv.org/abs/2111.09999 · HTML: https://mlsanity.com/paper/2111.09999/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Builds on 46 of its 73 references that are in the map.
- First posted on arXiv 19 Nov 2021.

## Builds on (most cited first, 30 of 46)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [Adversarial examples in the physical world](https://mlsanity.com/paper/1607.02533/) (2016)
- [Natural Adversarial Examples](https://mlsanity.com/paper/1907.07174/) (2019)
- [Certified Adversarial Robustness via Randomized Smoothing](https://mlsanity.com/paper/1902.02918/) (2019)
- [Colorful Image Colorization](https://mlsanity.com/paper/1603.08511/) (2016)
- [BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain](https://mlsanity.com/paper/1708.06733/) (2017)
- [Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning](https://mlsanity.com/paper/1712.05526/) (2017)
- [Practical Black-Box Attacks against Machine Learning](https://mlsanity.com/paper/1602.02697/) (2016)
- [Universal adversarial perturbations](https://mlsanity.com/paper/1610.08401/) (2016)
- [Stealing Machine Learning Models via Prediction APIs](https://mlsanity.com/paper/1609.02943/) (2016)
- [On Adaptive Attacks to Adversarial Example Defenses](https://mlsanity.com/paper/2002.08347/) (2020)
- [Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods](https://mlsanity.com/paper/1705.07263/) (2017)
- [Adversarial Patch](https://mlsanity.com/paper/1712.09665/) (2017)
- [Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples](https://mlsanity.com/paper/1605.07277/) (2016)
- [Approximating CNNs with Bag-of-local-Features models works surprisingly well on ImageNet](https://mlsanity.com/paper/1904.00760/) (2019)
- [Generating Adversarial Examples with Adversarial Networks](https://mlsanity.com/paper/1801.02610/) (2018)
- [DeepDriving: Learning Affordance for Direct Perception in Autonomous Driving](https://mlsanity.com/paper/1505.00256/) (2015)
- [Spatially Transformed Adversarial Examples](https://mlsanity.com/paper/1801.02612/) (2018)
- [Generating Natural Adversarial Examples](https://mlsanity.com/paper/1710.11342/) (2017)
- [Blind Backdoors in Deep Learning Models](https://mlsanity.com/paper/2005.03823/) (2020)
- [SentiNet: Detecting Localized Universal Attacks Against Deep Learning Systems](https://mlsanity.com/paper/1812.00292/) (2018)

## Built on by (most cited first, 30 of 36)

- [Physical Adversarial Attack meets Computer Vision: A Decade Survey](https://mlsanity.com/paper/2209.15179/) (2022)
- [DAP: A Dynamic Adversarial Patch for Evading Person Detectors](https://mlsanity.com/paper/2305.11618/) (2023)
- [Threats, Attacks, and Defenses in Machine Unlearning: A Survey](https://mlsanity.com/paper/2403.13682/) (2024)
- [A Survey on Physical Adversarial Attack in Computer Vision](https://mlsanity.com/paper/2209.14262/) (2022)
- [RFLA: A Stealthy Reflected Light Adversarial Attack in the Physical World](https://mlsanity.com/paper/2307.07653/) (2023)
- [ObjectSeeker: Certifiably Robust Object Detection against Patch Hiding Attacks via Patch-agnostic Masking](https://mlsanity.com/paper/2202.01811/) (2022)
- [Physical Adversarial Attacks For Camera-based Smart Systems: Current Trends, Categorization, Applications, Research Challenges, and Future Outlook](https://mlsanity.com/paper/2308.06173/) (2023)
- [Transferable Graph Backdoor Attack](https://mlsanity.com/paper/2207.00425/) (2022)
- [Towards Benchmarking and Assessing Visual Naturalness of Physical World Adversarial Attacks](https://mlsanity.com/paper/2305.12863/) (2023)
- [LLM Whisperer: An Inconspicuous Attack to Bias LLM Responses](https://mlsanity.com/paper/2406.04755/) (2024)
- [Feasibility of Inconspicuous GAN-generated Adversarial Patches against Object Detection](https://mlsanity.com/paper/2207.07347/) (2022)
- [Adversarial Infrared Blocks: A Multi-view Black-box Attack to Thermal Infrared Detectors in Physical World](https://mlsanity.com/paper/2304.10712/) (2023)
- [Diffusion to Confusion: Naturalistic Adversarial Patch Generation Based on Diffusion Model for Object Detector](https://mlsanity.com/paper/2307.08076/) (2023)
- [Adversarial Examples in the Physical World: A Survey](https://mlsanity.com/paper/2311.01473/) (2023)
- [Adversarial Infrared Curves: An Attack on Infrared Pedestrian Detectors in the Physical World](https://mlsanity.com/paper/2312.14217/) (2023)
- [Adversarial Neon Beam: A Light-based Physical Attack to DNNs](https://mlsanity.com/paper/2204.00853/) (2022)
- [A Comprehensive Study on the Robustness of Image Classification and Object Detection in Remote Sensing: Surveying and Benchmarking](https://mlsanity.com/paper/2306.12111/) (2023)
- [MVPatch: More Vivid Patch for Adversarial Camouflaged Attacks on Object Detectors in the Physical World](https://mlsanity.com/paper/2312.17431/) (2023)
- [Ti-Patch: Tiled Physical Adversarial Patch for no-reference video quality metrics](https://mlsanity.com/paper/2404.09961/) (2024)
- [BadPatch: Diffusion-Based Generation of Physical Adversarial Patches](https://mlsanity.com/paper/2412.01440/) (2024)
- [RAI4IoE: Responsible AI for Enabling the Internet of Energy](https://mlsanity.com/paper/2309.11691/) (2023)
- [Seeing Isn't Believing: Context-Aware Adversarial Patch Synthesis via Conditional GAN](https://mlsanity.com/paper/2509.22836/) (2025)
- [Adversarial Color Projection: A Projector-based Physical Attack to DNNs](https://mlsanity.com/paper/2209.09652/) (2022)
- [Adversarial Catoptric Light: An Effective, Stealthy and Robust Physical-World Attack to DNNs](https://mlsanity.com/paper/2209.11739/) (2022)
- [NSA: Naturalistic Support Artifact to Boost Network Confidence](https://mlsanity.com/paper/2307.14917/) (2023)
- [Fight Fire with Fire: Combating Adversarial Patch Attacks using Pattern-randomized Defensive Patches](https://mlsanity.com/paper/2311.06122/) (2023)
- [MAGIC: Mastering Physical Adversarial Generation in Context through Collaborative LLM Agents](https://mlsanity.com/paper/2412.08014/) (2024)
- [A Survey of Adversarial Defenses in Vision-based Systems: Categorization, Methods and Challenges](https://mlsanity.com/paper/2503.00384/) (2025)
- [DREAM: Domain-free Reverse Engineering Attributes of Black-box Model](https://mlsanity.com/paper/2307.10997/) (2023)
- [Enhancing Robust Representation in Adversarial Training: Alignment and Exclusion Criteria](https://mlsanity.com/paper/2310.03358/) (2023)

## Abstract (arXiv)

Deep neural networks are vulnerable to attacks from adversarial inputs and, more recently, Trojans to misguide or hijack the model's decision. We expose the existence of an intriguing class of spatially bounded, physically realizable, adversarial examples -- Universal NaTuralistic adversarial paTches -- we call TnTs, by exploring the superset of the spatially bounded adversarial example space and the natural input space within generative adversarial networks. Now, an adversary can arm themselves with a patch that is naturalistic, less malicious-looking, physically realizable, highly effective achieving high attack success rates, and universal. A TnT is universal because any input image captured with a TnT in the scene will: i) misguide a network (untargeted attack); or ii) force the network to make a malicious decision (targeted attack). Interestingly, now, an adversarial patch attacker has the potential to exert a greater level of control -- the ability to choose a location-independent, natural-looking patch as a trigger in contrast to being constrained to noisy perturbations -- an ability is thus far shown to be only possible with Trojan attack methods needing to interfere with the model building processes to embed a backdoor at the risk discovery; but, still realize a patch deployable in the physical world. Through extensive experiments on the large-scale visual classification task, ImageNet with evaluations across its entire validation set of 50,000 images, we demonstrate the realistic threat from TnTs and the robustness of the attack. We show a generalization of the attack to create patches achieving higher attack success rates than existing state-of-the-art methods. Our results show the generalizability of the attack to different visual classification tasks (CIFAR-10, GTSRB, PubFig) and multiple state-of-the-art deep neural networks such as WideResnet50, Inception-V3 and VGG-16.