How Does Bayes Error Limit Probabilistic Robust Accuracy
Not on the map. Why?
Abstract (arXiv)
Adversarial examples pose a security threat to many critical systems built on neural networks. Given that deterministic robustness often comes with significantly reduced accuracy, probabilistic robustness (i.e., the probability of having the same label with a vicinity is ≥ 1-kappa) has been proposed as a promising way of achieving robustness whilst maintaining accuracy. However, existing training methods for probabilistic robustness still experience non-trivial accuracy loss. It is unclear whether there is an upper bound on the accuracy when optimising towards probabilistic robustness, and whether there is a certain relationship between kappa and this bound. This work studies these problems from a Bayes error perspective. We find that while Bayes uncertainty does affect probabilistic robustness, its impact is smaller than that on deterministic robustness. This reduced Bayes uncertainty allows a higher upper bound on probabilistic robust accuracy than that on deterministic robust accuracy. Further, we prove that with optimal probabilistic robustness, each probabilistically robust input is also deterministically robust in a smaller vicinity. We also show that voting within the vicinity always improves probabilistic robust accuracy and the upper bound of probabilistic robust accuracy monotonically increases as kappa grows. Our empirical findings also align with our results.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 34 of its 54 references that are in the map. First posted on arXiv 23 May 2024.
Builds on
34 of 54 references.
References · 34
- 2014Explaining and Harnessing Adversarial Examplescited by 7,364
- 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
- 2013Intriguing properties of neural networkscited by 5,155
- 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
- 2015Domain-Adversarial Training of Neural Networkscited by 3,044
- 2016Adversarial examples in the physical worldcited by 1,769
- 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,545
- 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,517
- 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,340
- 2016Adversarial Machine Learning at Scalecited by 936
- 2020On Adaptive Attacks to Adversarial Example Defensescited by 555
- 2018Efficient Neural Network Robustness Certification with General Activation Functionscited by 445
22 more
- 2018Characterizing Adversarial Subspaces Using Local Intrinsic Dimensionalitycited by 282
- 2019Human uncertainty makes classification more robustcited by 235
- 2020SoK: Certified Robustness for Deep Neural Networkscited by 105
- 2020Certified Defenses for Adversarial Patchescited by 101
- 2019The Limitations of Adversarial Training and the Blind-Spot Attackcited by 63
- 2022Certified Training: Small Boxes are All You Needcited by 52
- 2021Fast Certified Robust Training with Short Warmupcited by 46
- 2022Probabilistically Robust Learning: Balancing Average- and Worst-case Performancecited by 37
- 2021RobOT: Robustness-Oriented Testing for Deep Learning Systemscited by 28
- 2022Is the Performance of My Deep Network Too Good to Be True? A Direct Approach to Estimating the Bayes Error in Binary Classificationcited by 26
- 2022PRoA: A Probabilistic Robustness Assessment against Functional Perturbationscited by 19
- 2021Evaluating State-of-the-Art Classification Models Against Bayes Optimalitycited by 16
- 2014Empirically Estimable Classification Bounds Based on a New Divergence Measurecited by 12
- 2016Maximum a Posteriori Estimators as a Limit of Bayes Estimatorscited by 12
- 2021Towards Practical Robustness Analysis for DNNs based on PAC-Model Learningcited by 12
- 2014Generalized Bhattacharyya and Chernoff upper bounds on Bayes error using quasi-arithmetic meanscited by 11
- 2019Learning to Benchmark: Determining Best Achievable Misclassification Error from Training Datacited by 11
- 2018Learning to Bound the Multi-class Bayes Errorcited by 9
- 2021Evaluating Bayes Error Estimators on Real-World Datasets with FeeBeecited by 8
- 2024Certified Robust Accuracy of Neural Networks Are Bounded due to Bayes Errorscited by 7
- 2021Certifying Emergency Landing for Safe Urban UAVcited by 6
- 2015Meta learning of bounds on the Bayes classifier errorcited by 2
Built on by
Citing papers per year: 2025: 2.
2 papers.