# Mitigating Uncertainty in Document Classification

arXiv: https://arxiv.org/abs/1907.07590 · HTML: https://mlsanity.com/paper/1907.07590/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Jul 2019.

## Built on by (most cited first, 22 of 22)

- [LM-Polygraph: Uncertainty Estimation for Language Models](https://mlsanity.com/paper/2311.07383/) (2023)
- [Benchmarking Uncertainty Quantification Methods for Large Language Models with LM-Polygraph](https://mlsanity.com/paper/2406.15627/) (2024)
- [Uncertainty in Natural Language Processing: Sources, Quantification, and Applications](https://mlsanity.com/paper/2306.04459/) (2023)
- [Uncertainty-Aware Reliable Text Classification](https://mlsanity.com/paper/2107.07114/) (2021)
- [(Why) Is My Prompt Getting Worse? Rethinking Regression Testing for Evolving LLM APIs](https://mlsanity.com/paper/2311.11123/) (2023)
- [A Head to Predict and a Head to Question: Pre-trained Uncertainty Quantification Heads for Hallucination Detection in LLM Outputs](https://mlsanity.com/paper/2505.08200/) (2025)
- [Token-Level Density-Based Uncertainty Quantification Methods for Eliciting Truthfulness of Large Language Models](https://mlsanity.com/paper/2502.14427/) (2025)
- [Incorporating Uncertainty in Learning to Defer Algorithms for Safe Computer-Aided Diagnosis](https://mlsanity.com/paper/2108.07392/) (2021)
- [Efficient Hallucination Detection for LLMs Using Uncertainty-Aware Attention Heads](https://mlsanity.com/paper/2505.20045/) (2025)
- [Certainly Uncertain: A Benchmark and Metric for Multimodal Epistemic and Aleatoric Awareness](https://mlsanity.com/paper/2407.01942/) (2024)
- [Do we really have to filter out random noise in pre-training data for language models?](https://mlsanity.com/paper/2502.06604/) (2025)
- [Uncertainty Estimation on Sequential Labeling via Uncertainty Transmission](https://mlsanity.com/paper/2311.08726/) (2023)
- [AutoTRIZ: Automating Engineering Innovation with TRIZ and Large Language Models](https://mlsanity.com/paper/2403.13002/) (2024)
- [Unconditional Truthfulness: Learning Unconditional Uncertainty of Large Language Models](https://mlsanity.com/paper/2408.10692/) (2024)
- [Uncertainty Estimation of Transformers' Predictions via Topological Analysis of the Attention Matrices](https://mlsanity.com/paper/2308.11295/) (2023)
- [SUN: Exploring Intrinsic Uncertainties in Text-to-SQL Parsers](https://mlsanity.com/paper/2209.06442/) (2022)
- [Semi-Supervised Dialogue Abstractive Summarization via High-Quality Pseudolabel Selection](https://mlsanity.com/paper/2403.04073/) (2024)
- [Entropy methods for the confidence assessment of probabilistic classification models](https://mlsanity.com/paper/2103.15157/) (2021)
- [Boosting Cross-Lingual Transfer via Self-Learning with Uncertainty Estimation](https://mlsanity.com/paper/2109.00194/) (2021)
- [Revisiting Softmax for Uncertainty Approximation in Text Classification](https://mlsanity.com/paper/2210.14037/) (2022)
- [AggMatch: Aggregating Pseudo Labels for Semi-Supervised Learning](https://mlsanity.com/paper/2201.10444/) (2022)
- [ALUM: Adversarial Data Uncertainty Modeling from Latent Model Uncertainty Compensation](https://mlsanity.com/paper/2303.16866/) (2023)

## Abstract (arXiv)

The uncertainty measurement of classifiers' predictions is especially important in applications such as medical diagnoses that need to ensure limited human resources can focus on the most uncertain predictions returned by machine learning models. However, few existing uncertainty models attempt to improve overall prediction accuracy where human resources are involved in the text classification task. In this paper, we propose a novel neural-network-based model that applies a new dropout-entropy method for uncertainty measurement. We also design a metric learning method on feature representations, which can boost the performance of dropout-based uncertainty methods with smaller prediction variance in accurate prediction trials. Extensive experiments on real-world data sets demonstrate that our method can achieve a considerable improvement in overall prediction accuracy compared to existing approaches. In particular, our model improved the accuracy from 0.78 to 0.92 when 30% of the most uncertain predictions were handed over to human experts in "20NewsGroup" data.