# Trainwreck: A damaging adversarial attack on image classifiers

arXiv: https://arxiv.org/abs/2311.14772 · HTML: https://mlsanity.com/paper/2311.14772/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 35 of its 46 references that are in the map.
- First posted on arXiv 24 Nov 2023.

## Builds on (most cited first, 30 of 35)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [Understanding Black-box Predictions via Influence Functions](https://mlsanity.com/paper/1703.04730/) (2017)
- [EfficientNetV2: Smaller Models and Faster Training](https://mlsanity.com/paper/2104.00298/) (2021)
- [BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain](https://mlsanity.com/paper/1708.06733/) (2017)
- [Adversarial Machine Learning at Scale](https://mlsanity.com/paper/1611.01236/) (2016)
- [Universal adversarial perturbations](https://mlsanity.com/paper/1610.08401/) (2016)
- [Reflection Backdoor: A Natural Backdoor Attack on Deep Neural Networks](https://mlsanity.com/paper/2007.02343/) (2020)
- [Improving Robustness using Generated Data](https://mlsanity.com/paper/2110.09468/) (2021)
- [Data Augmentation Can Improve Robustness](https://mlsanity.com/paper/2111.05328/) (2021)
- [Unlearnable Examples: Making Personal Data Unexploitable](https://mlsanity.com/paper/2101.04898/) (2021)
- [Adversarial Examples Make Strong Poisons](https://mlsanity.com/paper/2106.10807/) (2021)
- [Shadows can be Dangerous: Stealthy and Effective Physical-world Adversarial Attack by Natural Phenomenon](https://mlsanity.com/paper/2203.03818/) (2022)
- [Backdoor Attacks on Self-Supervised Learning](https://mlsanity.com/paper/2105.10123/) (2021)
- [BppAttack: Stealthy and Efficient Trojan Attacks against Deep Neural Networks via Image Quantization and Contrastive Adversarial Learning](https://mlsanity.com/paper/2205.13383/) (2022)
- [Learning to Confuse: Generating Training Time Adversarial Data with Auto-Encoder](https://mlsanity.com/paper/1905.09027/) (2019)
- [FIBA: Frequency-Injection based Backdoor Attack in Medical Image Analysis](https://mlsanity.com/paper/2112.01148/) (2021)
- [T-SEA: Transfer-based Self-Ensemble Attack on Object Detection](https://mlsanity.com/paper/2211.09773/) (2022)
- [Shape-invariant 3D Adversarial Point Clouds](https://mlsanity.com/paper/2203.04041/) (2022)
- [NICGSlowDown: Evaluating the Efficiency Robustness of Neural Image Caption Generation Models](https://mlsanity.com/paper/2203.15859/) (2022)
- [Towards Robust Rain Removal Against Adversarial Attacks: A Comprehensive Benchmark Analysis and Beyond](https://mlsanity.com/paper/2203.16931/) (2022)
- [Certified Patch Robustness via Smoothed Vision Transformers](https://mlsanity.com/paper/2110.07719/) (2021)
- [Towards Practical Certifiable Patch Defense with Vision Transformer](https://mlsanity.com/paper/2203.08519/) (2022)
- [Physically Realizable Natural-Looking Clothing Textures Evade Person Detectors via 3D Modeling](https://mlsanity.com/paper/2307.01778/) (2023)
- [Towards Certifying L-infinity Robustness using Neural Networks with L-inf-dist Neurons](https://mlsanity.com/paper/2102.05363/) (2021)
- [Dual-Key Multimodal Backdoors for Visual Question Answering](https://mlsanity.com/paper/2112.07668/) (2021)

## Abstract (arXiv)

Adversarial attacks are an important security concern for computer vision (CV). As CV models are becoming increasingly valuable assets in applied practice, disrupting them is emerging as a form of economic sabotage. This paper opens up the exploration of damaging adversarial attacks (DAAs) that seek to damage target CV models. DAAs are formalized by defining the threat model, the cost function DAAs maximize, and setting three requirements for success: potency, stealth, and customizability. As a pioneer DAA, this paper proposes Trainwreck, a train-time attack that conflates the data of similar classes in the training data using stealthy (ε ≤ 8/255) class-pair universal perturbations obtained from a surrogate model. Trainwreck is a black-box, transferable attack: it requires no knowledge of the target architecture, and a single poisoned dataset degrades the performance of any model trained on it. The experimental evaluation on CIFAR-10 and CIFAR-100 and various model architectures (EfficientNetV2, ResNeXt-101, and a finetuned ViT-L-16) demonstrates Trainwreck's efficiency. Trainwreck achieves similar or better potency compared to the data poisoning state of the art and is fully customizable by the poison rate parameter. Finally, data redundancy with hashing is identified as a reliable defense against Trainwreck or similar DAAs. The code is available at https://github.com/JanZahalka/trainwreck.