mlsanityData to 14 Oct 2026

To Predict or Not to Predict? Towards reliable uncertainty estimation in the presence of noise

arXiv 2603.07330 ↗First posted 7 Mar 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

This study examines the role of uncertainty estimation (UE) methods in multilingual text classification under noisy and non-topical conditions. Using a complex-vs-simple sentence classification task across several languages, we evaluate a range of UE techniques against a range of metrics to assess their contribution to making more robust predictions. Results indicate that while methods relying on softmax outputs remain competitive in high-resource in-domain settings, their reliability declines in low-resource or domain-shift scenarios. In contrast, Monte Carlo dropout approaches demonstrate consistently strong performance across all languages, offering more robust calibration, stable decision thresholds, and greater discriminative power even under adverse conditions. We further demonstrate the positive impact of UE on non-topical classification: abstaining from predicting the 10% most uncertain instances increases the macro F1 score from 0.81 to 0.85 in the Readme task. By integrating UE with trustworthiness metrics, this study provides actionable insights for developing more reliable NLP systems in real-world multilingual environments. See https://github.com/Nouran-Khallaf/To-Predict-or-Not-to-Predict

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 15 of its 27 references that are in the map. First posted on arXiv 7 Mar 2026.

Builds on

15 of 27 references.

References · 15

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2017On Calibration of Modern Neural Networkscited by 4,960
  3. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
  4. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
  5. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,385
  6. 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,485
  7. 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,328
  8. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,287
  9. 2017Selective Classification for Deep Neural Networkscited by 852
  10. 2020Calibrating Deep Neural Networks using Focal Losscited by 357
  11. 2021Revisiting Mahalanobis Distance for Transformer-Based Out-of-Domain Detectioncited by 70
  12. 2023A Survey on Out-of-Distribution Detection in NLPcited by 31
3 more
  1. 2023ReadMe++: Benchmarking Multilingual Language Models for Multi-Domain Readability Assessmentcited by 19
  2. 2025Uncertainty-aware abstention in medical diagnosis based on medical textscited by 6
  3. 2023Empirical Optimal Risk to Quantify Model Trustworthiness for Failure Detectioncited by 4

Built on by

Not cited yet.