# On Attacking Out-Domain Uncertainty Estimation in Deep Neural Networks

arXiv: https://arxiv.org/abs/2210.02191 · HTML: https://mlsanity.com/paper/2210.02191/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 32 references that are in the map.
- First posted on arXiv 3 Oct 2022.

## Builds on (most cited first, 17 of 17)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods](https://mlsanity.com/paper/1705.07263/) (2017)
- [Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awareness](https://mlsanity.com/paper/2006.10108/) (2020)
- [Why ReLU networks yield high-confidence predictions far away from the training data and how to mitigate the problem](https://mlsanity.com/paper/1812.05720/) (2018)
- [Adversarial Logit Pairing](https://mlsanity.com/paper/1803.06373/) (2018)
- [On Feature Collapse and Deep Kernel Learning for Single Forward Pass Uncertainty](https://mlsanity.com/paper/2102.11409/) (2021)
- [Towards Safe Autonomous Driving: Capture Uncertainty in the Deep Neural Network For Lidar 3D Vehicle Detection](https://mlsanity.com/paper/1804.05132/) (2018)
- [Adversarial Robustness on In- and Out-Distribution Improves Explainability](https://mlsanity.com/paper/2003.09461/) (2020)
- [Towards neural networks that provably know when they don't know](https://mlsanity.com/paper/1909.12180/) (2019)
- [Adversarial Examples, Uncertainty, and Transfer Testing Robustness in Gaussian Process Hybrid Deep Networks](https://mlsanity.com/paper/1707.02476/) (2017)
- [Disrupting Deep Uncertainty Estimation Without Harming Accuracy](https://mlsanity.com/paper/2110.13741/) (2021)
- [Provably Robust Detection of Out-of-distribution Data (almost) for free](https://mlsanity.com/paper/2106.04260/) (2021)
- [Adversarial Attack for Uncertainty Estimation: Identifying Critical Regions in Neural Networks](https://mlsanity.com/paper/2107.07618/) (2021)

## Built on by (most cited first, 4 of 4)

- [Towards Unveiling Predictive Uncertainty Vulnerabilities in the Context of the Right to Be Forgotten](https://mlsanity.com/paper/2508.07458/) (2025)
- [QA Domain Adaptation using Hidden Space Augmentation and Self-Supervised Contrastive Adaptation](https://mlsanity.com/paper/2210.10861/) (2022)
- [Attacking Bayes: On the Adversarial Robustness of Bayesian Neural Networks](https://mlsanity.com/paper/2404.19640/) (2024)
- [On the Robustness of Adversarial Training Against Uncertainty Attacks](https://mlsanity.com/paper/2410.21952/) (2024)

## Abstract (arXiv)

In many applications with real-world consequences, it is crucial to develop reliable uncertainty estimation for the predictions made by the AI decision systems. Targeting at the goal of estimating uncertainty, various deep neural network (DNN) based uncertainty estimation algorithms have been proposed. However, the robustness of the uncertainty returned by these algorithms has not been systematically explored. In this work, to raise the awareness of the research community on robust uncertainty estimation, we show that state-of-the-art uncertainty estimation algorithms could fail catastrophically under our proposed adversarial attack despite their impressive performance on uncertainty estimation. In particular, we aim at attacking the out-domain uncertainty estimation: under our attack, the uncertainty model would be fooled to make high-confident predictions for the out-domain data, which they originally would have rejected. Extensive experimental results on various benchmark image datasets show that the uncertainty estimated by state-of-the-art methods could be easily corrupted by our attack.