# Characterizing Data Point Vulnerability via Average-Case Robustness

arXiv: https://arxiv.org/abs/2307.13885 · HTML: https://mlsanity.com/paper/2307.13885/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 42 references that are in the map.
- First posted on arXiv 26 Jul 2023.

## Builds on (most cited first, 24 of 24)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [Certified Adversarial Robustness via Randomized Smoothing](https://mlsanity.com/paper/1902.02918/) (2019)
- [DeepFool: a simple and accurate method to fool deep neural networks](https://mlsanity.com/paper/1511.04599/) (2015)
- [Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift](https://mlsanity.com/paper/1906.02530/) (2019)
- [SmoothGrad: removing noise by adding noise](https://mlsanity.com/paper/1706.03825/) (2017)
- [Measuring Robustness to Natural Distribution Shifts in Image Classification](https://mlsanity.com/paper/2007.00644/) (2020)
- [Robustness via curvature regularization, and vice versa](https://mlsanity.com/paper/1811.09716/) (2018)
- [(Certified!!) Adversarial Robustness for Free!](https://mlsanity.com/paper/2206.10550/) (2022)
- [Which Explanation Should I Choose? A Function Approximation Perspective to Characterizing Post Hoc Explanations](https://mlsanity.com/paper/2206.01254/) (2022)
- [Knowledge Transfer with Jacobian Matching](https://mlsanity.com/paper/1803.00443/) (2018)
- [Fairness Through Robustness: Investigating Robustness Disparity in Deep Learning](https://mlsanity.com/paper/2006.12621/) (2020)
- [Towards the Unification and Robustness of Perturbation and Gradient Based Explanations](https://mlsanity.com/paper/2102.10618/) (2021)
- [Probabilistically Robust Recourse: Navigating the Trade-offs between Costs and Robustness in Algorithmic Recourse](https://mlsanity.com/paper/2203.06768/) (2022)
- [Probabilistically Robust Learning: Balancing Average- and Worst-case Performance](https://mlsanity.com/paper/2202.01136/) (2022)
- [Certifying Confidence via Randomized Smoothing](https://mlsanity.com/paper/2009.08061/) (2020)
- [An Effective Baseline for Robustness to Distributional Shift](https://mlsanity.com/paper/2105.07107/) (2021)
- [Explicit Tradeoffs between Adversarial and Natural Distributional Robustness](https://mlsanity.com/paper/2209.07592/) (2022)
- [Which Models have Perceptually-Aligned Gradients? An Explanation via Off-Manifold Robustness](https://mlsanity.com/paper/2305.19101/) (2023)
- [Efficiently Training Low-Curvature Neural Networks](https://mlsanity.com/paper/2206.07144/) (2022)
- [Probabilistic Verification and Reachability Analysis of Neural Networks via Semidefinite Programming](https://mlsanity.com/paper/1910.04249/) (2019)

## Built on by (most cited first, 3 of 3)

- [T-SHIRT: Token-Selective Hierarchical Data Selection for Instruction Tuning](https://mlsanity.com/paper/2506.01317/) (2025)
- [Robust Fine-Tuning from Non-Robust Pretrained Models: Mitigating Suboptimal Transfer With Epsilon-Scheduling](https://mlsanity.com/paper/2509.23325/) (2025)
- [Dynamics to decision: A mathematical theory of Lyapunov spectra and decision boundaries in deep classifiers](https://mlsanity.com/paper/2609.39190/) (2026)

## Abstract (arXiv)

Studying the robustness of machine learning models is important to ensure consistent model behaviour across real-world settings. To this end, adversarial robustness is a standard framework, which views robustness of predictions through a binary lens: either a worst-case adversarial misclassification exists in the local region around an input, or it does not. However, this binary perspective does not account for the degrees of vulnerability, as data points with a larger number of misclassified examples in their neighborhoods are more vulnerable. In this work, we consider a complementary framework for robustness, called average-case robustness, which measures the fraction of points in a local region that provides consistent predictions. However, computing this quantity is hard, as standard Monte Carlo approaches are inefficient especially for high-dimensional inputs. In this work, we propose the first analytical estimators for average-case robustness for multi-class classifiers. We show empirically that our estimators are accurate and efficient for standard deep learning models and demonstrate their usefulness for identifying vulnerable data points, as well as quantifying robustness bias of models. Overall, our tools provide a complementary view to robustness, improving our ability to characterize model behaviour.