# Interpreting and Evaluating Neural Network Robustness

arXiv: https://arxiv.org/abs/1905.04270 · HTML: https://mlsanity.com/paper/1905.04270/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 May 2019.

## Built on by (most cited first, 19 of 19)

- [Trustworthy Graph Neural Networks: Aspects, Methods and Trends](https://mlsanity.com/paper/2205.07424/) (2022)
- [Prior and Posterior Networks: A Survey on Evidential Deep Learning Methods For Uncertainty Estimation](https://mlsanity.com/paper/2110.03051/) (2021)
- [On the Effectiveness of Lipschitz-Driven Rehearsal in Continual Learning](https://mlsanity.com/paper/2210.06443/) (2022)
- [AdvRush: Searching for Adversarially Robust Neural Architectures](https://mlsanity.com/paper/2108.01289/) (2021)
- [SAFARI: Versatile and Efficient Evaluations for Robustness of Interpretability](https://mlsanity.com/paper/2208.09418/) (2022)
- [RES: A Robust Framework for Guiding Visual Explanation](https://mlsanity.com/paper/2206.13413/) (2022)
- [A.I. Robustness: a Human-Centered Perspective on Technological Challenges and Opportunities](https://mlsanity.com/paper/2210.08906/) (2022)
- [Combating Bilateral Edge Noise for Robust Link Prediction](https://mlsanity.com/paper/2311.01196/) (2023)
- [Understanding and Measuring Robustness of Multimodal Learning](https://mlsanity.com/paper/2112.12792/) (2021)
- [Data Models for Dataset Drift Controls in Machine Learning With Optical Images](https://mlsanity.com/paper/2211.02578/) (2022)
- [Enabling All In-Edge Deep Learning: A Literature Review](https://mlsanity.com/paper/2204.03326/) (2022)
- [Aspect-Based Sentiment Analysis for Open-Ended HR Survey Responses](https://mlsanity.com/paper/2402.04812/) (2024)
- [How Robust is your Fair Model? Exploring the Robustness of Diverse Fairness Strategies](https://mlsanity.com/paper/2207.04581/) (2022)
- [Adversarial attacks and defenses on ML- and hardware-based IoT device fingerprinting and identification](https://mlsanity.com/paper/2212.14677/) (2022)
- [Boosting Verified Training for Robust Image Classifications via Abstraction](https://mlsanity.com/paper/2303.11552/) (2023)
- [Open Set Relation Extraction via Unknown-Aware Training](https://mlsanity.com/paper/2306.04950/) (2023)
- [Probing Human Visual Robustness with Neurally-Guided Deep Neural Networks](https://mlsanity.com/paper/2405.02564/) (2024)
- [Understanding the Interplay between Privacy and Robustness in Federated Learning](https://mlsanity.com/paper/2106.07033/) (2021)
- [Boosting-based Construction of BDDs for Linear Threshold Functions and Its Application to Verification of Neural Networks](https://mlsanity.com/paper/2306.05211/) (2023)

## Abstract (arXiv)

Recently, adversarial deception becomes one of the most considerable threats to deep neural networks. However, compared to extensive research in new designs of various adversarial attacks and defenses, the neural networks' intrinsic robustness property is still lack of thorough investigation. This work aims to qualitatively interpret the adversarial attack and defense mechanism through loss visualization, and establish a quantitative metric to evaluate the neural network model's intrinsic robustness. The proposed robustness metric identifies the upper bound of a model's prediction divergence in the given domain and thus indicates whether the model can maintain a stable prediction. With extensive experiments, our metric demonstrates several advantages over conventional adversarial testing accuracy based robustness estimation: (1) it provides a uniformed evaluation to models with different structures and parameter scales; (2) it over-performs conventional accuracy based robustness estimation and provides a more reliable evaluation that is invariant to different test settings; (3) it can be fast generated without considerable testing cost.