# Holistic Approach to Measure Sample-level Adversarial Vulnerability and its Utility in Building Trustworthy Systems

arXiv: https://arxiv.org/abs/2205.02604 · HTML: https://mlsanity.com/paper/2205.02604/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 38 references that are in the map.
- First posted on arXiv 5 May 2022.

## Builds on (most cited first, 25 of 25)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [FitNets: Hints for Thin Deep Nets](https://mlsanity.com/paper/1412.6550/) (2014)
- [ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness](https://mlsanity.com/paper/1811.12231/) (2018)
- [DeepFool: a simple and accurate method to fool deep neural networks](https://mlsanity.com/paper/1511.04599/) (2015)
- [Adversarial Examples Are Not Bugs, They Are Features](https://mlsanity.com/paper/1905.02175/) (2019)
- [The Limitations of Deep Learning in Adversarial Settings](https://mlsanity.com/paper/1511.07528/) (2015)
- [Object Detection with Deep Learning: A Review](https://mlsanity.com/paper/1807.05511/) (2018)
- [High Frequency Component Helps Explain the Generalization of Convolutional Neural Networks](https://mlsanity.com/paper/1905.13545/) (2019)
- [Characterising Bias in Compressed Models](https://mlsanity.com/paper/2010.03058/) (2020)
- [To be Robust or to be Fair: Towards Fairness in Adversarial Training](https://mlsanity.com/paper/2010.06121/) (2020)
- [Deep learning in remote sensing: a review](https://mlsanity.com/paper/1710.03959/) (2017)
- [Adversarial vulnerability for any classifier](https://mlsanity.com/paper/1802.08686/) (2018)
- [Knowledge Transfer with Jacobian Matching](https://mlsanity.com/paper/1803.00443/) (2018)
- [Fairness Through Robustness: Investigating Robustness Disparity in Deep Learning](https://mlsanity.com/paper/2006.12621/) (2020)
- [Recent Advances in Deep Learning for Object Detection](https://mlsanity.com/paper/1908.03673/) (2019)
- [Knowledge Distillation with Adversarial Samples Supporting Decision Boundary](https://mlsanity.com/paper/1805.05532/) (2018)
- [Robustness May Be at Odds with Fairness: An Empirical Study on Class-wise Accuracy](https://mlsanity.com/paper/2010.13365/) (2020)
- [Attacks on State-of-the-Art Face Recognition using Attentional Adversarial Attack Generative Network](https://mlsanity.com/paper/1811.12026/) (2018)
- [Fooling a Real Car with Adversarial Traffic Signs](https://mlsanity.com/paper/1907.00374/) (2019)
- [Towards Frequency-Based Explanation for Robust CNN](https://mlsanity.com/paper/2005.03141/) (2020)
- [Generalizability vs. Robustness: Adversarial Examples for Medical Imaging](https://mlsanity.com/paper/1804.00504/) (2018)
- [Few-shot learning of neural networks from scratch by pseudo example optimization](https://mlsanity.com/paper/1802.03039/) (2018)

## Built on by (most cited first, 2 of 2)

- [UnbiasedNets: A Dataset Diversification Framework for Robustness Bias Alleviation in Neural Networks](https://mlsanity.com/paper/2302.12538/) (2023)
- [Measuring Equality in Machine Learning Security Defenses: A Case Study in Speech Recognition](https://mlsanity.com/paper/2302.08973/) (2023)

## Abstract (arXiv)

Adversarial attack perturbs an image with an imperceptible noise, leading to incorrect model prediction. Recently, a few works showed inherent bias associated with such attack (robustness bias), where certain subgroups in a dataset (e.g. based on class, gender, etc.) are less robust than others. This bias not only persists even after adversarial training, but often results in severe performance discrepancies across these subgroups. Existing works characterize the subgroup's robustness bias by only checking individual sample's proximity to the decision boundary. In this work, we argue that this measure alone is not sufficient and validate our argument via extensive experimental analysis. It has been observed that adversarial attacks often corrupt the high-frequency components of the input image. We, therefore, propose a holistic approach for quantifying adversarial vulnerability of a sample by combining these different perspectives, i.e., degree of model's reliance on high-frequency features and the (conventional) sample-distance to the decision boundary. We demonstrate that by reliably estimating adversarial vulnerability at the sample level using the proposed holistic metric, it is possible to develop a trustworthy system where humans can be alerted about the incoming samples that are highly likely to be misclassified at test time. This is achieved with better precision when our holistic metric is used over individual measures. To further corroborate the utility of the proposed holistic approach, we perform knowledge distillation in a limited-sample setting. We observe that the student network trained with the subset of samples selected using our combined metric performs better than both the competing baselines, viz., where samples are selected randomly or based on their distances to the decision boundary.