# Towards Deep Learning Models Resistant to Large Perturbations

arXiv: https://arxiv.org/abs/2003.13370 · HTML: https://mlsanity.com/paper/2003.13370/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Mar 2020.

## Built on by (most cited first, 8 of 8)

- [Efficient and Effective Augmentation Strategy for Adversarial Training](https://mlsanity.com/paper/2210.15318/) (2022)
- [Scaling Adversarial Training to Large Perturbation Bounds](https://mlsanity.com/paper/2210.09852/) (2022)
- [Fast Adversarial Training with Smooth Convergence](https://mlsanity.com/paper/2308.12857/) (2023)
- [Adversarial Training: A Survey](https://mlsanity.com/paper/2410.15042/) (2024)
- [Bridged Adversarial Training](https://mlsanity.com/paper/2108.11135/) (2021)
- [FrameShield: Adversarially Robust Video Anomaly Detection](https://mlsanity.com/paper/2510.21532/) (2025)
- [Blacksmith: Fast Adversarial Training of Vision Transformers via a Mixture of Single-step and Multi-step Methods](https://mlsanity.com/paper/2310.18975/) (2023)
- [Towards Assessing Deep Learning Test Input Generators](https://mlsanity.com/paper/2504.02329/) (2025)

## Abstract (arXiv)

Adversarial robustness has proven to be a required property of machine learning algorithms. A key and often overlooked aspect of this problem is to try to make the adversarial noise magnitude as large as possible to enhance the benefits of the model robustness. We show that the well-established algorithm called "adversarial training" fails to train a deep neural network given a large, but reasonable, perturbation magnitude. In this paper, we propose a simple yet effective initialization of the network weights that makes learning on higher levels of noise possible. We next evaluate this idea rigorously on MNIST (ε up to ≈ 0.40) and CIFAR10 (ε up to ≈ 32/255) datasets assuming the ℓ∞ attack model. Additionally, in order to establish the limits of ε in which the learning is feasible, we study the optimal robust classifier assuming full access to the joint data and label distribution. Then, we provide some theoretical results on the adversarial accuracy for a simple multi-dimensional Bernoulli distribution, which yields some insights on the range of feasible perturbations for the MNIST dataset.