# Uncertainty-Aware Reliable Text Classification

arXiv: https://arxiv.org/abs/2107.07114 · HTML: https://mlsanity.com/paper/2107.07114/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 54 references that are in the map.
- First posted on arXiv 15 Jul 2021.

## Builds on (most cited first, 30 of 35)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks](https://mlsanity.com/paper/1610.02136/) (2016)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [A large annotated corpus for learning natural language inference](https://mlsanity.com/paper/1508.05326/) (2015)
- [Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift](https://mlsanity.com/paper/1906.02530/) (2019)
- [Deep Anomaly Detection with Outlier Exposure](https://mlsanity.com/paper/1812.04606/) (2018)
- [Evidential Deep Learning to Quantify Classification Uncertainty](https://mlsanity.com/paper/1806.01768/) (2018)
- [Universal Adversarial Triggers for Attacking and Analyzing NLP](https://mlsanity.com/paper/1908.07125/) (2019)
- [Adversarial Examples for Evaluating Reading Comprehension Systems](https://mlsanity.com/paper/1707.07328/) (2017)
- [Why ReLU networks yield high-confidence predictions far away from the training data and how to mitigate the problem](https://mlsanity.com/paper/1812.05720/) (2018)
- [On Mixup Training: Improved Calibration and Predictive Uncertainty for Deep Neural Networks](https://mlsanity.com/paper/1905.11001/) (2019)
- [Pretrained Transformers Improve Out-of-Distribution Robustness](https://mlsanity.com/paper/2004.06100/) (2020)
- [Multi30K: Multilingual English-German Image Descriptions](https://mlsanity.com/paper/1605.00459/) (2016)
- [Disentangling Adversarial Robustness and Generalization](https://mlsanity.com/paper/1812.00740/) (2018)
- [Multiplicative Normalizing Flows for Variational Bayesian Neural Networks](https://mlsanity.com/paper/1703.01961/) (2017)
- [Flipout: Efficient Pseudo-Independent Weight Perturbations on Mini-Batches](https://mlsanity.com/paper/1803.04386/) (2018)
- [Deep Active Learning for Named Entity Recognition](https://mlsanity.com/paper/1707.05928/) (2017)
- [Deep Bayesian Active Learning for Natural Language Processing: Results of a Large-Scale Empirical Study](https://mlsanity.com/paper/1808.05697/) (2018)
- [Quantifying Uncertainties in Natural Language Processing Tasks](https://mlsanity.com/paper/1811.07253/) (2018)
- [Towards neural networks that provably know when they don't know](https://mlsanity.com/paper/1909.12180/) (2019)
- [Out-of-domain Detection for Natural Language Understanding in Dialog Systems](https://mlsanity.com/paper/1909.03862/) (2019)

## Built on by (most cited first, 23 of 23)

- [Delving into Out-of-Distribution Detection with Vision-Language Representations](https://mlsanity.com/paper/2211.13445/) (2022)
- [Uncertainty in Natural Language Processing: Sources, Quantification, and Applications](https://mlsanity.com/paper/2306.04459/) (2023)
- [Word-Sequence Entropy: Towards Uncertainty Estimation in Free-Form Medical Question Answering Applications and Beyond](https://mlsanity.com/paper/2402.14259/) (2024)
- [Addressing Uncertainty in LLMs to Enhance Reliability in Generative AI](https://mlsanity.com/paper/2411.02381/) (2024)
- [Uncertainty Estimation on Sequential Labeling via Uncertainty Transmission](https://mlsanity.com/paper/2311.08726/) (2023)
- [SeTAR: Out-of-Distribution Detection with Selective Low-Rank Approximation](https://mlsanity.com/paper/2406.12629/) (2024)
- [Easier to Mislead Than to Correct: Harmful and Beneficial Revision in LLM Conformity](https://mlsanity.com/paper/2606.01637/) (2026)
- [SUN: Exploring Intrinsic Uncertainties in Text-to-SQL Parsers](https://mlsanity.com/paper/2209.06442/) (2022)
- [E-NER: Evidential Deep Learning for Trustworthy Named Entity Recognition](https://mlsanity.com/paper/2305.17854/) (2023)
- [Mysteries of the Deep: Role of Intermediate Representations in Out of Distribution Detection](https://mlsanity.com/paper/2510.05782/) (2025)
- [Controllable Fake Document Infilling for Cyber Deception](https://mlsanity.com/paper/2210.09917/) (2022)
- [Uncertainty-aware Self-training for Low-resource Neural Sequence Labeling](https://mlsanity.com/paper/2302.08659/) (2023)
- [Uncertainty-aware Parameter-Efficient Self-training for Semi-supervised Language Understanding](https://mlsanity.com/paper/2310.13022/) (2023)
- [Text Classification Under Class Distribution Shift: A Survey](https://mlsanity.com/paper/2502.12965/) (2025)
- [Safety-Flag: A Unified Benchmark for the Reliability and Calibration of LLM Content Moderators](https://mlsanity.com/paper/2609.19072/) (2026)
- [Defining and Evaluating Decision and Composite Risk in Language Models Applied to Natural Language Inference](https://mlsanity.com/paper/2408.01935/) (2024)
- [Coarse-to-Fine Open-Set Graph Node Classification with Large Language Models](https://mlsanity.com/paper/2512.16244/) (2025)
- [Identifying Technical Debt and Its Types Across Diverse Software Projects Issues](https://mlsanity.com/paper/2408.09128/) (2024)
- [Identifying Trustworthiness Challenges in Deep Learning Models for Continental-Scale Water Quality Prediction](https://mlsanity.com/paper/2503.09947/) (2025)
- [Mastering the Minority: An Uncertainty-guided Multi-Expert Framework for Challenging-tailed Sequence Learning](https://mlsanity.com/paper/2603.15708/) (2026)
- [Can I Take Another Dose? Evaluating LLM Decision-Making Under Temporal Uncertainty in OTC Dosing QA](https://mlsanity.com/paper/2606.04262/) (2026)
- [When Surface Form Changes Moderation Decisions: A Paired Study of Code-Mixed Workflow Instability](https://mlsanity.com/paper/2606.05654/) (2026)
- [Subtype Robustness Is Not Just Accuracy: Calibration Under Unseen Subtype Shift](https://mlsanity.com/paper/2608.00928/) (2026)

## Abstract (arXiv)

Deep neural networks have significantly contributed to the success in predictive accuracy for classification tasks. However, they tend to make over-confident predictions in real-world settings, where domain shifting and out-of-distribution (OOD) examples exist. Most research on uncertainty estimation focuses on computer vision because it provides visual validation on uncertainty quality. However, few have been presented in the natural language process domain. Unlike Bayesian methods that indirectly infer uncertainty through weight uncertainties, current evidential uncertainty-based methods explicitly model the uncertainty of class probabilities through subjective opinions. They further consider inherent uncertainty in data with different root causes, vacuity (i.e., uncertainty due to a lack of evidence) and dissonance (i.e., uncertainty due to conflicting evidence). In our paper, we firstly apply evidential uncertainty in OOD detection for text classification tasks. We propose an inexpensive framework that adopts both auxiliary outliers and pseudo off-manifold samples to train the model with prior knowledge of a certain class, which has high vacuity for OOD samples. Extensive empirical experiments demonstrate that our model based on evidential uncertainty outperforms other counterparts for detecting OOD examples. Our approach can be easily deployed to traditional recurrent neural networks and fine-tuned pre-trained transformers.