GREAT Score: Global Robustness Evaluation of Adversarial Perturbation using Generative Models
Not on the map. Why?
Abstract (arXiv)
Current studies on adversarial robustness mainly focus on aggregating local robustness results from a set of data samples to evaluate and rank different models. However, the local statistics may not well represent the true global robustness of the underlying unknown data distribution. To address this challenge, this paper makes the first attempt to present a new framework, called GREAT Score , for global robustness evaluation of adversarial perturbation using generative models. Formally, GREAT Score carries the physical meaning of a global statistic capturing a mean certified attack-proof perturbation level over all samples drawn from a generative model. For finite-sample evaluation, we also derive a probabilistic guarantee on the sample complexity and the difference between the sample mean and the true mean. GREAT Score has several advantages: (1) Robustness evaluations using GREAT Score are efficient and scalable to large models, by sparing the need of running adversarial attacks. In particular, we show high correlation and significantly reduced computation cost of GREAT Score when compared to the attack-based model ranking on RobustBench (Croce,et. al. 2021). (2) The use of generative models facilitates the approximation of the unknown data distribution. In our ablation study with different generative adversarial networks (GANs), we observe consistency between global robustness evaluation and the quality of GANs. (3) GREAT Score can be used for remote auditing of privacy-sensitive black-box models, as demonstrated by our robustness evaluation on several online facial recognition services.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 47 of its 69 references that are in the map. First posted on arXiv 19 Apr 2023.
Builds on
47 of 69 references.
References · 47
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2020Denoising Diffusion Probabilistic Modelscited by 20,851
- 2014Explaining and Harnessing Adversarial Examplescited by 7,287
- 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
- 2022Generative Adversarial Networkscited by 5,603
- 2013Intriguing properties of neural networkscited by 5,114
- 2014Deep Learning Face Attributes in the Wildcited by 3,679
- 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
- 2016Wide Residual Networkscited by 3,057
- 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
- 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
- 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,875
35 more
- 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
- 2016Identity Mappings in Deep Residual Networkscited by 1,960
- 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
- 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,442
- 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,324
- 2016Least Squares Generative Adversarial Networkscited by 1,000
- 2018Self-Attention Generative Adversarial Networkscited by 809
- 2017Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networkscited by 739
- 2020Fast is better than free: Revisiting adversarial trainingcited by 723
- 2019Interpreting the Latent Space of GANs for Semantic Face Editingcited by 691
- 2019Square Attack: a query-efficient black-box adversarial attack via random searchcited by 682
- 2020RobustBench: a standardized adversarial robustness benchmarkcited by 659
- 2017Provable defenses against adversarial examples via the convex outer adversarial polytopecited by 564
- 2020Overfitting in adversarially robust deep learningcited by 504
- 2018Efficient Neural Network Robustness Certification with General Activation Functionscited by 439
- 2020Do Adversarially Robust ImageNet Models Transfer Better?cited by 315
- 2019Efficient and Accurate Estimation of Lipschitz Constants for Deep Neural Networkscited by 290
- 2020Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examplescited by 275
- 2017Geometric GANcited by 232
- 2021Fixing Data Augmentation to Improve Adversarial Robustnesscited by 201
- 2017Formal Guarantees on the Robustness of a Classifier against Adversarial Manipulationcited by 182
- 2018Evaluating the Robustness of Neural Networks: An Extreme Value Theory Approachcited by 181
- 2018MMA Training: Direct Input Space Margin Maximization through Adversarial Trainingcited by 171
- 2017EAD: Elastic-Net Attacks to Deep Neural Networks via Adversarial Examplescited by 154
- 2021Robust Learning Meets Generative Models: Can Proxy Distributions Improve Adversarial Robustness?cited by 113
- 2018Decoupling Direction and Norm for Efficient Gradient-Based L2 Adversarial Attacks and Defensescited by 109
- 2021Globally-Robust Neural Networkscited by 106
- 2018How Well Generative Adversarial Networks Learn Distributionscited by 78
- 2020Adversarial Robustness on In- and Out-Distribution Improves Explainabilitycited by 73
- 2021Rebooting ACGAN: Auxiliary Classifier GANs with Stable Trainingcited by 57
- 2022Probabilistically Robust Learning: Balancing Average- and Worst-case Performancecited by 36
- 2020Statistical guarantees for generative models without dominationcited by 27
- 2022How many perturbations break this model? Evaluating robustness beyond adversarial accuracycited by 9
- 2018Global Robustness Evaluation of Deep Neural Networks with Provable Guarantees for the L0 Normcited by 4
- 2021Some Hoeffding- and Bernstein-type Concentration Inequalitiescited by 1
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.