mlsanityData to 14 Oct 2026

Uncertainty Estimation for the Open-Set Text Classification systems

arXiv 2604.08560 ↗First posted 17 Mar 2026cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Accurate uncertainty estimation is essential for building robust and trustworthy recognition systems. In this paper, we consider the open-set text classification (OSTC) task - and uncertainty estimation for it. For OSTC a text sample should be classified as one of the existing classes or rejected as unknown. To account for the different uncertainty types encountered in OSTC, we adapt the Holistic Uncertainty Estimation (HolUE) method for the text domain. Our approach addresses two major causes of prediction errors in text recognition systems: text uncertainty that stems from ill formulated queries and gallery uncertainty that is related the ambiguity of data distribution. By capturing these sources, it becomes possible to predict when the system will make a recognition error. We propose a new OSTC benchmark and conduct extensive experiments on a wide range of data, utilizing the authorship attribution, intent and topic classification datasets. HolUE achieves 40-365% improvement in Prediction Rejection Ratio (PRR) over the quality-based SCF baseline across datasets: 365% on Yahoo Answers (0.79 vs 0.17 at FPIR 0.1), 347% on DBPedia (0.85 vs 0.19), 240% on PAN authorship attribution (0.51 vs 0.15 at FPIR 0.5), and 40% on CLINC150 intent classification (0.73 vs~0.52). We make public our code and protocols https://github.com/Leonid-Erlygin/text_uncertainty.git

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 19 of its 38 references that are in the map. First posted on arXiv 17 Mar 2026.

Builds on

19 of 38 references.

References · 19

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2017On Calibration of Modern Neural Networkscited by 4,960
  3. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
  4. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
  5. 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,352
  6. 2015Towards Open Set Deep Networkscited by 538
  7. 2019An Evaluation Dataset for Intent Classification and Out-of-Scope Predictioncited by 386
  8. 2018Recent Advances in Open Set Recognition: A Surveycited by 310
  9. 2019Probabilistic Face Embeddingscited by 173
  10. 2023LM-Polygraph: Uncertainty Estimation for Language Modelscited by 131
  11. 2017DOC: Deep Open Classification of Text Documentscited by 109
  12. 2019Deep Unknown Intent Detection with Margin Losscited by 66
7 more
  1. 2021Open-world Machine Learning: Applications, Challenges, and Opportunitiescited by 34
  2. 2017Toward Open-Set Face Recognitioncited by 29
  3. 2024VoxBlink2: A 100K+ Speaker Recognition Corpus and the Open-Set Speaker-Identification Benchmarkcited by 26
  4. 2025Hallucination Detection in LLMs with Topological Divergence on Attention Graphscited by 19
  5. 2022Stating Comparison Score Uncertainty and Verification Decision Confidence Towards Transparent Face Recognitioncited by 9
  6. 2019Open Set Authorship Attribution toward Demystifying Victorian Periodicalscited by 3
  7. 2024Holistic Uncertainty Estimation For Open-Set Recognitioncited by 3

Built on by

Not cited yet.