# Evaluating Robustness of Predictive Uncertainty Estimation: Are Dirichlet-based Models Reliable?

arXiv: https://arxiv.org/abs/2010.14986 · HTML: https://mlsanity.com/paper/2010.14986/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Oct 2020.

## Built on by (most cited first, 30 of 39)

- [Trusted Multi-View Classification with Dynamic Evidential Fusion](https://mlsanity.com/paper/2204.11423/) (2022)
- [Quantifying Aleatoric and Epistemic Uncertainty in Machine Learning: Are Conditional Entropy and Mutual Information Appropriate Measures?](https://mlsanity.com/paper/2209.03302/) (2022)
- [Graph Posterior Network: Bayesian Predictive Uncertainty for Node Classification](https://mlsanity.com/paper/2110.14012/) (2021)
- [Prior and Posterior Networks: A Survey on Evidential Deep Learning Methods For Uncertainty Estimation](https://mlsanity.com/paper/2110.03051/) (2021)
- [Pitfalls of Epistemic Uncertainty Quantification through Loss Minimisation](https://mlsanity.com/paper/2203.06102/) (2022)
- [Uncertainty in Natural Language Processing: Sources, Quantification, and Applications](https://mlsanity.com/paper/2306.04459/) (2023)
- [On Second-Order Scoring Rules for Epistemic Uncertainty Quantification](https://mlsanity.com/paper/2301.12736/) (2023)
- [Is Epistemic Uncertainty Faithfully Represented by Evidential Deep Learning Methods?](https://mlsanity.com/paper/2402.09056/) (2024)
- [Learn to Accumulate Evidence from All Training Samples: Theory and Practice](https://mlsanity.com/paper/2306.11113/) (2023)
- [Disentangling Epistemic and Aleatoric Uncertainty in Reinforcement Learning](https://mlsanity.com/paper/2206.01558/) (2022)
- [Towards Open Set Video Anomaly Detection](https://mlsanity.com/paper/2208.11113/) (2022)
- [Disrupting Deep Uncertainty Estimation Without Harming Accuracy](https://mlsanity.com/paper/2110.13741/) (2021)
- [Random-Set Neural Networks (RS-NN)](https://mlsanity.com/paper/2307.05772/) (2023)
- [Provably Robust Detection of Out-of-distribution Data (almost) for free](https://mlsanity.com/paper/2106.04260/) (2021)
- [Uncertainty Quantification for Machine Learning in Healthcare: A Survey](https://mlsanity.com/paper/2505.02874/) (2025)
- [Natural Posterior Network: Deep Bayesian Uncertainty for Exponential Family Distributions](https://mlsanity.com/paper/2105.04471/) (2021)
- [Density-Softmax: Efficient Test-time Model for Uncertainty Estimation and Robustness under Distribution Shifts](https://mlsanity.com/paper/2302.06495/) (2023)
- [Uncertainty Estimation for Molecules: Desiderata and Methods](https://mlsanity.com/paper/2306.14916/) (2023)
- [Dense Uncertainty Estimation](https://mlsanity.com/paper/2110.06427/) (2021)
- [Position: Epistemic uncertainty estimation methods are fundamentally incomplete](https://mlsanity.com/paper/2505.23506/) (2025)
- [On the Calibration of Probabilistic Classifier Sets](https://mlsanity.com/paper/2205.10082/) (2022)
- [Epistemic Artificial Intelligence is Essential for Machine Learning Models to Truly 'Know When They Do Not Know'](https://mlsanity.com/paper/2505.04950/) (2025)
- [E-NER: Evidential Deep Learning for Trustworthy Named Entity Recognition](https://mlsanity.com/paper/2305.17854/) (2023)
- [A calibration test for evaluating set-based epistemic uncertainty representations](https://mlsanity.com/paper/2502.16299/) (2025)
- [Towards Certification of Uncertainty Calibration under Adversarial Attacks](https://mlsanity.com/paper/2405.13922/) (2024)
- [Linear Opinion Pooling for Uncertainty Quantification on Graphs](https://mlsanity.com/paper/2406.04041/) (2024)
- [Dynamic ensemble selection based on Deep Neural Network Uncertainty Estimation for Adversarial Robustness](https://mlsanity.com/paper/2308.00346/) (2023)
- [Attacking Bayes: On the Adversarial Robustness of Bayesian Neural Networks](https://mlsanity.com/paper/2404.19640/) (2024)
- [Robust Adversarial Quantification via Conflict-Aware Evidential Deep Learning](https://mlsanity.com/paper/2506.05937/) (2025)
- [Generalized Regularized Evidential Deep Learning Models: Theory and Comprehensive Evaluation](https://mlsanity.com/paper/2512.23753/) (2025)

## Abstract (arXiv)

Dirichlet-based uncertainty (DBU) models are a recent and promising class of uncertainty-aware models. DBU models predict the parameters of a Dirichlet distribution to provide fast, high-quality uncertainty estimates alongside with class predictions. In this work, we present the first large-scale, in-depth study of the robustness of DBU models under adversarial attacks. Our results suggest that uncertainty estimates of DBU models are not robust w.r.t. three important tasks: (1) indicating correctly and wrongly classified samples; (2) detecting adversarial examples; and (3) distinguishing between in-distribution (ID) and out-of-distribution (OOD) data. Additionally, we explore the first approaches to make DBU models more robust. While adversarial training has a minor effect, our median smoothing based approach significantly increases robustness of DBU models.