mlsanityData to 7 Oct 2026

On Continuity of Robust and Accurate Classifiers

arXiv 2309.17048 ↗First posted 29 Sep 2023cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

The reliability of a learning model is key to the successful deployment of machine learning in various applications. However, it is difficult to describe the phenomenon due to the complicated nature of the problems in machine learning. It has been shown that adversarial training can improve the robustness of the hypothesis. However, this improvement usually comes at the cost of decreased performance on natural samples. Hence, it has been suggested that robustness and accuracy of a hypothesis are at odds with each other. In this paper, we put forth the alternative proposal that it is the continuity of a hypothesis that is incompatible with its robustness and accuracy in many of these scenarios. In other words, a continuous function cannot effectively learn the optimal robust hypothesis. We introduce a framework for a rigorous study of harmonic and holomorphic hypothesis in learning theory terms and provide empirical evidence that continuous hypotheses do not perform as well as discontinuous hypotheses in some common machine learning tasks. From a practical point of view, our results suggests that a robust and accurate learning rule would train different continuous hypotheses for different regions of the domain. From a theoretical perspective, our analysis explains the adversarial examples phenomenon in these situations as a conflict between the continuity of a sequence of functions and its uniform convergence to a discontinuous function. Given that many of the contemporary machine learning models are continuous functions, it is important to theoretically study the continuity of robust and accurate classifiers as it is consequential in their construction, analysis and evaluation.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 42 of its 73 references that are in the map. First posted on arXiv 29 Sep 2023.

Builds on

42 of 73 references.

References · 42

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  3. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  4. 2013Intriguing properties of neural networkscited by 5,114
  5. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  6. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  7. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
  8. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,442
  9. 2019Adversarial Examples Are Not Bugs, They Are Featurescited by 950
  10. 2018Robustness May Be at Odds with Accuracycited by 873
  11. 2019Square Attack: a query-efficient black-box adversarial attack via random searchcited by 682
  12. 2017Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methodscited by 542
30 more
  1. 2016Delving into Transferable Adversarial Examples and Black-box Attackscited by 530
  2. 2018Deep Learning for Classical Japanese Literaturecited by 415
  3. 2019Minimally distorted Adversarial Examples with a Fast Adaptive Boundary Attackcited by 333
  4. 2017Parseval Networks: Improving Robustness to Adversarial Examplescited by 288
  5. 2021Recent Advances in Adversarial Training for Adversarial Robustnesscited by 276
  6. 2013Learning Factored Representations in a Deep Mixture of Expertscited by 275
  7. 2017Improving the Adversarial Robustness and Interpretability of Deep Neural Networks by Regularizing their Input Gradientscited by 236
  8. 2017EMNIST: an extension of MNIST to handwritten letterscited by 212
  9. 2017On the (Statistical) Detection of Adversarial Examplescited by 180
  10. 2020Torchattacks: A PyTorch Repository for Adversarial Attackscited by 156
  11. 2020Implicit Gradient Regularizationcited by 145
  12. 2018Is Robustness the Cost of Accuracy? -- A Comprehensive Study on the Robustness of 18 Deep Image Classification Modelscited by 135
  13. 2019Adversarial Training Can Hurt Generalizationcited by 131
  14. 2016Technical Report on the CleverHans v2.1.0 Adversarial Examples Librarycited by 123
  15. 2019Adversarial Robustness Against the Union of Multiple Perturbation Modelscited by 88
  16. 2016Adversarial Examples Detection in Deep Networks with Convolutional Filter Statisticscited by 77
  17. 2016A Boundary Tilting Persepective on the Phenomenon of Adversarial Examplescited by 70
  18. 2021Adversarial Example Detection for DNN Models: A Review and Experimental Comparisoncited by 51
  19. 2021Mind the box: l1-APGD for sparse adversarial attacks on image classifierscited by 50
  20. 2018Limitations of the Lipschitz constant as a defense against adversarial examplescited by 48
  21. 2015A Unified Gradient Regularization Family for Adversarial Examplescited by 43
  22. 2020Provable tradeoffs in adversarially robust classificationcited by 39
  23. 2021GradDiv: Adversarial Robustness of Randomized Neural Networks via Gradient Diversity Regularizationcited by 34
  24. 2017Gradient Regularization Improves Accuracy of Discriminative Modelscited by 26
  25. 2019Transfer of Adversarial Robustness Between Perturbation Typescited by 25
  26. 2020Friedrichs Learning: Weak Solutions of Partial Differential Equations via Deep Learningcited by 18
  27. 2019Training Provably Robust Models by Polyhedral Envelope Regularizationcited by 6
  28. 2023Revisiting the Trade-off between Accuracy and Robustness via Weight Distribution of Filterscited by 4
  29. 2018Detecting Adversarial Examples in Convolutional Neural Networkscited by 3
  30. 2019Reducing Adversarial Example Transferability Using Gradient Regularizationcited by 1

Built on by

Not cited yet.