# Label Smoothing and Logit Squeezing: A Replacement for Adversarial Training?

arXiv: https://arxiv.org/abs/1910.11585 · HTML: https://mlsanity.com/paper/1910.11585/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Oct 2019.

## Built on by (most cited first, 18 of 18)

- [RobustART: Benchmarking Robustness on Architecture Design and Training Techniques](https://mlsanity.com/paper/2109.05211/) (2021)
- [Adversarial Robustness under Long-Tailed Distribution](https://mlsanity.com/paper/2104.02703/) (2021)
- [Reliably fast adversarial training via latent adversarial perturbation](https://mlsanity.com/paper/2104.01575/) (2021)
- [Exploring the Impact of Temperature Scaling in Softmax for Classification and Adversarial Robustness](https://mlsanity.com/paper/2502.20604/) (2025)
- [NoiLIn: Improving Adversarial Training and Correcting Stereotype of Noisy Labels](https://mlsanity.com/paper/2105.14676/) (2021)
- [In and Out-of-Domain Text Adversarial Robustness via Label Smoothing](https://mlsanity.com/paper/2212.10258/) (2022)
- [Countermeasures Against Adversarial Examples in Radio Signal Classification](https://mlsanity.com/paper/2407.06796/) (2024)
- [Class-Level Logit Perturbation](https://mlsanity.com/paper/2209.05668/) (2022)
- [Similarity of Neural Architectures using Adversarial Attack Transferability](https://mlsanity.com/paper/2210.11407/) (2022)
- [A Hybrid Training-time and Run-time Defense Against Adversarial Attacks in Modulation Classification](https://mlsanity.com/paper/2407.06807/) (2024)
- [Understanding the Logit Distributions of Adversarially-Trained Deep Neural Networks](https://mlsanity.com/paper/2108.12001/) (2021)
- [New Paradigm of Adversarial Training: Releasing Accuracy-Robustness Trade-Off via Dummy Class](https://mlsanity.com/paper/2410.12671/) (2024)
- [Guidance Through Surrogate: Towards a Generic Diagnostic Attack](https://mlsanity.com/paper/2212.14875/) (2022)
- [Breaking Boundaries: Balancing Performance and Robustness in Deep Wireless Traffic Forecasting](https://mlsanity.com/paper/2311.09790/) (2023)
- [Label Augmentation for Neural Networks Robustness](https://mlsanity.com/paper/2408.01977/) (2024)
- [Adversarial Robustness Overestimation and Instability in TRADES](https://mlsanity.com/paper/2410.07675/) (2024)
- [ODG-Q: Robust Quantization via Online Domain Generalization](https://mlsanity.com/paper/2210.08701/) (2022)
- [Batch-in-Batch: a new adversarial training framework for initial perturbation and sample selection](https://mlsanity.com/paper/2406.04070/) (2024)

## Abstract (arXiv)

Adversarial training is one of the strongest defenses against adversarial attacks, but it requires adversarial examples to be generated for every mini-batch during optimization. The expense of producing these examples during training often precludes adversarial training from use on complex image datasets. In this study, we explore the mechanisms by which adversarial training improves classifier robustness, and show that these mechanisms can be effectively mimicked using simple regularization methods, including label smoothing and logit squeezing. Remarkably, using these simple regularization methods in combination with Gaussian noise injection, we are able to achieve strong adversarial robustness -- often exceeding that of adversarial training -- using no adversarial examples.