# Learn to Accumulate Evidence from All Training Samples: Theory and Practice

arXiv: https://arxiv.org/abs/2306.11113 · HTML: https://mlsanity.com/paper/2306.11113/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 36 references that are in the map.
- First posted on arXiv 19 Jun 2023.

## Builds on (most cited first, 22 of 22)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images](https://mlsanity.com/paper/1412.1897/) (2014)
- [Evidential Deep Learning to Quantify Classification Uncertainty](https://mlsanity.com/paper/1806.01768/) (2018)
- [Predictive Uncertainty Estimation via Prior Networks](https://mlsanity.com/paper/1802.10501/) (2018)
- [Deep Evidential Regression](https://mlsanity.com/paper/1910.02600/) (2019)
- [Meta-Baseline: Exploring Simple Meta-Learning for Few-Shot Learning](https://mlsanity.com/paper/2003.04390/) (2020)
- [Posterior Network: Uncertainty Estimation without OOD Samples via Density-Based Pseudo-Counts](https://mlsanity.com/paper/2006.09239/) (2020)
- [Reverse KL-Divergence Training of Prior Networks: Improved Uncertainty and Adversarial Robustness](https://mlsanity.com/paper/1905.13472/) (2019)
- [Evidential Deep Learning for Open Set Action Recognition](https://mlsanity.com/paper/2107.10161/) (2021)
- [Uncertainty Aware Semi-Supervised Learning on Graph Data](https://mlsanity.com/paper/2010.12783/) (2020)
- [Uncertainty in Neural Networks: Approximately Bayesian Ensembling](https://mlsanity.com/paper/1810.05546/) (2018)
- [DropConnect Is Effective in Modeling Uncertainty of Bayesian Deep Networks](https://mlsanity.com/paper/1906.04569/) (2019)
- [Evaluating Robustness of Predictive Uncertainty Estimation: Are Dirichlet-based Models Reliable?](https://mlsanity.com/paper/2010.14986/) (2020)
- [Towards Trustworthy Predictions from Deep Neural Networks with Fast Adversarial Calibration](https://mlsanity.com/paper/2012.10923/) (2020)
- [Vision Transformers in 2022: An Update on Tiny ImageNet](https://mlsanity.com/paper/2205.10660/) (2022)
- [Evidential Conditional Neural Processes](https://mlsanity.com/paper/2212.00131/) (2022)
- [Multidimensional Belief Quantification for Label-Efficient Meta-Learning](https://mlsanity.com/paper/2203.12768/) (2022)

## Built on by (most cited first, 29 of 29)

- [Think Twice Before Selection: Federated Evidential Active Learning for Medical Image Analysis with Domain Shifts](https://mlsanity.com/paper/2312.02567/) (2023)
- [A Comprehensive Survey on Evidential Deep Learning and Its Applications](https://mlsanity.com/paper/2409.04720/) (2024)
- [Conformalized Credal Regions for Classification with Ambiguous Ground Truth](https://mlsanity.com/paper/2411.04852/) (2024)
- [Revisiting Essential and Nonessential Settings of Evidential Deep Learning](https://mlsanity.com/paper/2410.00393/) (2024)
- [Uncertainty Estimation by Flexible Evidential Deep Learning](https://mlsanity.com/paper/2510.18322/) (2025)
- [Provable Uncertainty Decomposition via Higher-Order Calibration](https://mlsanity.com/paper/2412.18808/) (2024)
- [Credal and Interval Deep Evidential Classifications](https://mlsanity.com/paper/2512.05526/) (2025)
- [Evidential Semantic Mapping in Off-road Environments with Uncertainty-aware Bayesian Kernel Inference](https://mlsanity.com/paper/2403.14138/) (2024)
- [Deep Evidential Learning for Bayesian Quantile Regression](https://mlsanity.com/paper/2308.10650/) (2023)
- [Multimodal Learning with Uncertainty Quantification based on Discounted Belief Fusion](https://mlsanity.com/paper/2412.18024/) (2024)
- [Weakly-Supervised Residual Evidential Learning for Multi-Instance Uncertainty Estimation](https://mlsanity.com/paper/2405.04405/) (2024)
- [Epistemic Generative Adversarial Networks](https://mlsanity.com/paper/2603.18348/) (2026)
- [Label Calibration in Source Free Domain Adaptation](https://mlsanity.com/paper/2501.07072/) (2025)
- [Uncertainty-Aware Likelihood Ratio Estimation for Pixel-Wise Out-of-Distribution Detection](https://mlsanity.com/paper/2508.00587/) (2025)
- [Rethinking Approximate Gaussian Inference in Classification](https://mlsanity.com/paper/2502.03366/) (2025)
- [Robust Adversarial Quantification via Conflict-Aware Evidential Deep Learning](https://mlsanity.com/paper/2506.05937/) (2025)
- [Generalized Regularized Evidential Deep Learning Models: Theory and Comprehensive Evaluation](https://mlsanity.com/paper/2512.23753/) (2025)
- [Evidential Transformation Network: Turning Pretrained Models into Evidential Models for Post-hoc Uncertainty Estimation](https://mlsanity.com/paper/2604.08627/) (2026)
- [TPFL: A Trustworthy Personalized Federated Learning Framework via Subjective Logic](https://mlsanity.com/paper/2410.12316/) (2024)
- [Uncertainty Quantification with Proper Scoring Rules: Adjusting Measures to Prediction Tasks](https://mlsanity.com/paper/2505.22538/) (2025)
- [Generalized Evidential Deep Learning: From a Bayesian Perspective](https://mlsanity.com/paper/2605.25599/) (2026)
- [DepthWeave-KV: Token-Adaptive Cross-Layer Residual Factorization for Long-Context KV Cache Compression](https://mlsanity.com/paper/2607.06523/) (2026)
- [Evidential Uncertainty Probes for Graph Neural Networks](https://mlsanity.com/paper/2503.08097/) (2025)
- [Ensemble-Based Dirichlet Modeling for Predictive Uncertainty and Selective Classification](https://mlsanity.com/paper/2604.06032/) (2026)
- [Courtroom Analogy: New Perspective on Uncertainty-Aware Classification](https://mlsanity.com/paper/2605.25616/) (2026)
- [Prompt-Adapter Context Routing for Parameter-Efficient Multi-Shot Long Video Extrapolation](https://mlsanity.com/paper/2607.06481/) (2026)
- [FreqDepthKV: Frequency-Guided Depth Sharing for Robust KV Cache Compression in Long-Context LLM Inference](https://mlsanity.com/paper/2607.06519/) (2026)
- [Robust Evidential Learning Through Latent Consistency](https://mlsanity.com/paper/2610.01384/) (2026)
- [Evidential Multimodal Transformers with Dempster-Shafer Fusion for Emotion Recognition](https://mlsanity.com/paper/2610.13013/) (2026)

## Abstract (arXiv)

Evidential deep learning, built upon belief theory and subjective logic, offers a principled and computationally efficient way to turn a deterministic neural network uncertainty-aware. The resultant evidential models can quantify fine-grained uncertainty using the learned evidence. To ensure theoretically sound evidential models, the evidence needs to be non-negative, which requires special activation functions for model training and inference. This constraint often leads to inferior predictive performance compared to standard softmax models, making it challenging to extend them to many large-scale datasets. To unveil the real cause of this undesired behavior, we theoretically investigate evidential models and identify a fundamental limitation that explains the inferior performance: existing evidential activation functions create zero evidence regions, which prevent the model to learn from training samples falling into such regions. A deeper analysis of evidential activation functions based on our theoretical underpinning inspires the design of a novel regularizer that effectively alleviates this fundamental limitation. Extensive experiments over many challenging real-world datasets and settings confirm our theoretical findings and demonstrate the effectiveness of our proposed approach.