mlsanityData to 14 Oct 2026

Average Certified Radius is a Poor Metric for Randomized Smoothing

arXiv 2410.06895 ↗First posted 9 Oct 2024cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

Randomized smoothing (RS) is popular for providing certified robustness guarantees against adversarial attacks. The average certified radius (ACR) has emerged as a widely used metric for tracking progress in RS. However, in this work, for the first time we show that ACR is a poor metric for evaluating robustness guarantees provided by RS. We theoretically prove not only that a trivial classifier can have arbitrarily large ACR, but also that ACR is extremely sensitive to improvements on easy samples. In addition, the comparison using ACR has a strong dependence on the certification budget. Empirically, we confirm that existing training strategies, though improving ACR, reduce the model's robustness on hard samples consistently. To strengthen our findings, we propose strategies, including explicitly discarding hard samples, reweighing the dataset with approximate certified radius, and extreme optimization for easy samples, to replicate the progress in RS training and even achieve the state-of-the-art ACR on CIFAR-10, without training for robustness on the full data distribution. Overall, our results suggest that ACR has introduced a strong undesired bias to the field, and its application should be discontinued in RS. Finally, we suggest using the empirical distribution of pA, the accuracy of the base model on noisy data, as an alternative metric for RS.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 33 of its 38 references that are in the map. First posted on arXiv 9 Oct 2024.

Builds on

33 of 38 references.

References · 33

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  3. 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,340
  4. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,268
  5. 2018Certified Robustness to Adversarial Examples with Differential Privacycited by 468
  6. 2019Provably Robust Deep Learning via Adversarially Trained Smoothed Classifierscited by 330
  7. 2018On the Effectiveness of Interval Bound Propagation for Training Verifiably Robust Modelscited by 328
  8. 2018Certified Adversarial Robustness with Additive Noisecited by 180
  9. 2022(Certified!!) Adversarial Robustness for Free!cited by 132
  10. 2020MACER: Attack-free and Scalable Robust Training via Maximizing Certified Radiuscited by 114
  11. 2020Consistency Regularization for Certified Robustness of Smoothed Classifierscited by 73
  12. 2022Certified Training: Small Boxes are All You Needcited by 52
21 more
  1. 2021SmoothMix: Training Confidence-calibrated Smoothed Classifiers for Certified Robustnesscited by 49
  2. 2021Boosting Randomized Smoothing with Variance Reduced Classifierscited by 47
  3. 2021Fast Certified Robust Training with Short Warmupcited by 46
  4. 2021On the Certified Robustness for Ensemble Models and Beyondcited by 40
  5. 2020Data-Dependent Randomized Smoothingcited by 37
  6. 2022DensePure: Understanding Diffusion Models towards Adversarial Robustnesscited by 33
  7. 2023Understanding Certified Training with Interval Bound Propagationcited by 26
  8. 2021Intriguing Properties of Input-dependent Randomized Smoothingcited by 24
  9. 2023Expressive Losses for Verified Robustness via Convex Combinationscited by 23
  10. 2022Double Sampling Randomized Smoothingcited by 22
  11. 2022Denoising Masked AutoEncoders Help Robust Classificationcited by 21
  12. 2023DiffSmooth: Certifiably Robust Learning via Diffusion Models and Local Smoothingcited by 20
  13. 2022Double Bubble, Toil and Trouble: Enhancing Certified Robustness through Transitivitycited by 15
  14. 2020Adversarial robustness via robust low rank representationscited by 14
  15. 2024CTBENCH: A Library and Benchmark for Certified Trainingcited by 14
  16. 2021Input-Specific Robustness Certification for Randomized Smoothingcited by 11
  17. 2023TAPS: Connecting Certified and Adversarial Trainingcited by 11
  18. 2022Confidence-aware Training of Smoothed Classifiers for Certified Robustnesscited by 9
  19. 2023Multi-scale Diffusion Denoised Smoothingcited by 9
  20. 2022Accelerating Certified Robustness Training via Knowledge Transfercited by 7
  21. 2024Consistency Purification: Effective and Efficient Diffusion Purification towards Certified Robustnesscited by 2

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Dual Randomized Smoothing: Beyond Global Noise Variancecited by 0