Data to 7 Oct 2026

Theoretically Principled Trade-off between Robustness and Accuracy

arXiv 1901.08573 ↗First posted 24 Jan 2019cs.LG stat.MLcited by 1,510

Not on the map. Why?

Cited by 53 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We identify a trade-off between robustness and accuracy that serves as a guiding principle in the design of defenses against adversarial examples. Although this problem has been widely studied empirically, much remains unknown concerning the theory underlying this trade-off. In this work, we decompose the prediction error for adversarial examples (robust error) as the sum of the natural (classification) error and boundary error, and provide a differentiable upper bound using the theory of classification-calibrated loss, which is shown to be the tightest possible upper bound uniform over all probability distributions and measurable predictors. Inspired by our theoretical analysis, we also design a new defense method, TRADES, to trade adversarial robustness off against accuracy. Our proposed algorithm performs well experimentally in real-world datasets. The methodology is the foundation of our entry to the NeurIPS 2018 Adversarial Vision Challenge in which we won the 1st place out of ~2,000 submissions, surpassing the runner-up approach by 11.41% in terms of mean ℓ2 perturbation distance.

Key facts. Cited by 1,510 papers in the map (data through 7 Oct 2026). First posted on arXiv 24 Jan 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 288; 2022: 317; 2023: 333; 2024: 266; 2025: 202; 2026: 104.

1,510 papers.

Sort:

By area:

2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
2022Diffusion Models for Adversarial Purificationcited by 469
2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attackscited by 434
2022Test-Time Prompt Tuning for Zero-Shot Generalization in Vision-Language Modelscited by 379
2021Unsolved Problems in ML Safetycited by 291
2021Recent Advances in Adversarial Training for Adversarial Robustnesscited by 276
2023Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Modelscited by 260
2021Enhancing the Transferability of Adversarial Attacks through Variance Tuningcited by 242
2021Improving Robustness using Generated Datacited by 233
2023Better Diffusion Models Further Improve Adversarial Trainingcited by 207
2021Trustworthy AI: From Principles to Practicescited by 206
2023Cross-Entropy Loss Functions: Theoretical Analysis and Applicationscited by 206

38 more

2021Fixing Data Augmentation to Improve Adversarial Robustnesscited by 201
2021Data Augmentation Can Improve Robustnesscited by 193
2021Unlearnable Examples: Making Personal Data Unexploitablecited by 188
2023How Robust is Google's Bard to Adversarial Image Attacks?cited by 177
2021The Values Encoded in Machine Learning Researchcited by 172
2024Defensive Unlearning with Adversarial Training for Robust Concept Erasure in Diffusion Modelscited by 164
2021Admix: Enhancing the Transferability of Adversarial Attackscited by 155
2022Towards Adversarial Attack on Vision-Language Pre-training Modelscited by 142
2021Robust Reinforcement Learning on State Observations with Learned Optimal Adversarycited by 133
2021Adversarial purification with Score-based generative modelscited by 132
2022Understanding Zero-Shot Adversarial Robustness for Large-Scale Modelscited by 117
2021Can contrastive learning avoid shortcut solutions?cited by 114
2021Robust Learning Meets Generative Models: Can Proxy Distributions Improve Adversarial Robustness?cited by 113
2022Robustness and Accuracy Could Be Reconcilable by (Proper) Definitioncited by 109
2021Measure and Improve Robustness in NLP Models: A Surveycited by 107
2021Globally-Robust Neural Networkscited by 106
2021On the Adversarial Robustness of Vision Transformerscited by 104
2021AdaMatch: A Unified Approach to Semi-Supervised Learning and Domain Adaptationcited by 104
2024Efficient Adversarial Training in LLMs with Continuous Attackscited by 104
2021ResLT: Residual Learning for Long-tailed Recognitioncited by 96
2022LAS-AT: Adversarial Training with Learnable Attack Strategycited by 94
2021Trustworthy AI: A Computational Perspectivecited by 93
2022Guided Diffusion Model for Adversarial Purificationcited by 90
2022The Unreasonable Effectiveness of Random Pruning: Return of the Most Naive Baseline for Sparse Trainingcited by 85
2023A Comprehensive Study on Robustness of Image Classification Models: Benchmarking and Rethinkingcited by 85
2024Defending Against Unforeseen Failure Modes with Latent Adversarial Trainingcited by 82
2023Revisiting Adversarial Training for ImageNet: Architectures, Training and Generalization across Threat Modelscited by 79
2023Robust Classification via a Single Diffusion Modelcited by 79
2021RobustART: Benchmarking Robustness on Architecture Design and Training Techniquescited by 78
2021Revisiting Adversarial Robustness Distillation: Robust Soft Labels Make Student Bettercited by 77
2021Improving Adversarial Robustness via Channel-wise Activation Suppressingcited by 76
2021When Does Contrastive Learning Preserve Adversarial Robustness from Pretraining to Finetuning?cited by 76
2022Frequency-driven Imperceptible Adversarial Attack on Semantic Similaritycited by 76
2021Revisiting and Advancing Fast Adversarial Training Through The Lens of Bi-Level Optimizationcited by 75
2022Improving Vision Transformers by Revisiting High-frequency Componentscited by 75
2021Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networkscited by 73
2021Towards Robustness Against Natural Language Word Substitutionscited by 68
2022The Effects of Regularization and Data Augmentation are Class Dependentcited by 67

50 most cited of 1,510 All 1,510 (JSON)