Quantifying Uncertainties in Natural Language Processing Tasks
Not on the map. Why?
Abstract (arXiv)
Reliable uncertainty quantification is a first step towards building explainable, transparent, and accountable artificial intelligent systems. Recent progress in Bayesian deep learning has made such quantification realizable. In this paper, we propose novel methods to study the benefits of characterizing model and data uncertainties for natural language processing (NLP) tasks. With empirical experiments on sentiment analysis, named entity recognition, and language modeling using convolutional and recurrent neural network models, we show that explicitly modeling uncertainties is not only necessary to measure output confidence levels, but also useful at enhancing model performances in various NLP tasks.
Key facts. Cited by 79 papers in the map (data through 7 Oct 2026). First posted on arXiv 18 Nov 2018.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 13; 2022: 11; 2023: 18; 2024: 19; 2025: 16; 2026: 2.
79 papers.
Sort:
By area:
- 2023Robots That Ask For Help: Uncertainty Alignment for Large Language Model Plannerscited by 298
- 2021On Hallucination and Predictive Uncertainty in Conditional Language Generationcited by 176
- 2024Kernel Language Entropy: Fine-grained Uncertainty Quantification for LLMs from Semantic Similaritiescited by 156
- 2024A Survey of Uncertainty Estimation in LLMs: Theory Meets Practicecited by 54
- 2023Active Instruction Tuning: Improving Cross-Task Generalization by Training on Prompt Sensitive Taskscited by 53
- 2024Conformal Prediction for Natural Language Processing: A Surveycited by 52
- 2024Uncertainty Quantification for In-Context Learning of Large Language Modelscited by 45
- 2024MARS: Meaning-Aware Response Scoring for Uncertainty Estimation in Generative LLMscited by 45
- 2023Uncertainty in Natural Language Processing: Sources, Quantification, and Applicationscited by 44
- 2022Scalable Uncertainty Quantification for Deep Operator Networks using Randomized Priorscited by 39
- 2022Weakly Supervised Data Augmentation Through Prompting for Dialogue Understandingcited by 32
- 2021Distributed NLI: Learning to Predict Human Opinion Distributions for Language Reasoningcited by 31
38 more
- 2023CLARA: Classifying and Disambiguating User Commands for Reliable Interactive Robotic Agentscited by 28
- 2025Trust Me, I'm Wrong: LLMs Hallucinate with Certainty Despite Knowing the Answercited by 26
- 2023On Uncertainty Calibration and Selective Generation in Probabilistic Neural Summarization: A Benchmark Studycited by 25
- 2021A Survey on Epistemic (Model) Uncertainty in Supervised Learning: Recent Advances and Applicationscited by 24
- 2021Uncertainty-Aware Reliable Text Classificationcited by 23
- 2021Uncertainty-Aware Balancing for Multilingual and Multi-Domain Neural Machine Translation Trainingcited by 19
- 2023Hyperbolic Active Learning for Semantic Segmentation under Domain Shiftcited by 18
- 2024Probabilistically Correct Language-based Multi-Robot Planning using Conformal Predictioncited by 18
- 2024Addressing Uncertainty in LLMs to Enhance Reliability in Generative AIcited by 15
- 2025Herd Behavior: Investigating Peer Influence in LLM-based Multi-Agent Systemscited by 15
- 2023CAVEN: An Embodied Conversational Agent for Efficient Audio-Visual Navigation in Noisy Environmentscited by 14
- 2024Uncertainty is Fragile: Manipulating Uncertainty in Large Language Modelscited by 13
- 2025Uncertainty Quantification and Decomposition for LLM-based Recommendationcited by 12
- 2022Calibration of Machine Reading Systems at Scalecited by 11
- 2023Toward Reliable Human Pose Forecasting with Uncertaintycited by 11
- 2023Efficient Uncertainty Quantification and Reduction for Over-Parameterized Neural Networkscited by 10
- 2021Trust It or Not: Confidence-Guided Automatic Radiology Report Generationcited by 9
- 2023Understanding Difficulty-based Sample Weighting with a Universal Difficulty Measurecited by 9
- 2025Grammars of Formal Uncertainty: When to Trust LLMs in Automated Reasoning Taskscited by 9
- 2021Attribution of Predictive Uncertainties in Classification Modelscited by 8
- 2023Reliable Gradient-free and Likelihood-free Prompt Tuningcited by 8
- 2022Efficient, Uncertainty-based Moderation of Neural Networks Text Classifierscited by 7
- 2022SUN: Exploring Intrinsic Uncertainties in Text-to-SQL Parserscited by 7
- 2023Bag of Tricks for In-Distribution Calibration of Pretrained Transformerscited by 7
- 2023Uncertainty-Aware Bootstrap Learning for Joint Extraction on Distantly-Supervised Datacited by 7
- 2024Uncertainty-boosted Robust Video Activity Anticipationcited by 7
- 2021Giving Commands to a Self-Driving Car: How to Deal with Uncertain Situations?cited by 6
- 2025The Consistency Hypothesis in Uncertainty Quantification for Large Language Modelscited by 6
- 2025HACK: Hallucinations Along Certainty and Knowledge Axescited by 6
- 2021Tracing Halpha Fibrils through Bayesian Deep Learningcited by 5
- 2021Estimating Predictive Uncertainty Under Program Data Distribution Shiftcited by 5
- 2024Self-Evaluation of Large Language Model based on Glass-box Featurescited by 5
- 2024Enhancing Trust in Large Language Models via Uncertainty-Calibrated Fine-Tuningcited by 5
- 2021Exploring Bayesian Deep Learning for Urgent Instructor Intervention Need in MOOC Forumscited by 4
- 2021Boosting Cross-Lingual Transfer via Self-Learning with Uncertainty Estimationcited by 4
- 2022Exploring the Learning Difficulty of Data Theory and Measurecited by 4
- 2024Taking Action Towards Graceful Interaction: The Effects of Performing Actions on Modelling Policies for Instruction Clarification Requestscited by 4
- 2025AdaptiveLog: An Adaptive Log Analysis Framework with the Collaboration of Large and Small Language Modelcited by 4
50 most cited of 79 All 79 (JSON)
- 2026Credal Concept Bottleneck Models for Epistemic-Aleatoric Uncertainty Decomposition27 Apr 2026
- 2026Uncertainty and Fairness Awareness in LLM-Based Recommendation Systems31 Jan 2026
- 2025Don't Throw Away Your Beams: Improving Consistency-based Uncertainties in LLMs via Beam Search10 Dec 2025
- 2025UncertaintyZoo: A Unified Toolkit for Quantifying Predictive Uncertainty in Deep Learning Systems6 Dec 2025
- 2025HACK: Hallucinations Along Certainty and Knowledge Axes28 Oct 2025
- 2025Multi-Modal Fact-Verification Framework for Reducing Hallucinations in Large Language Models26 Oct 2025
- 2025Confidence Calibration in Large Language Model-Based Entity Matching23 Sep 2025
- 2025Are you sure? Measuring models bias in content moderation through uncertainty21 Sep 2025
- 2025Non-exchangeable Conformal Prediction for Temporal Graph Neural Networks2 Jul 2025
- 2025The Consistency Hypothesis in Uncertainty Quantification for Large Language Models27 Jun 2025
- 2025Herd Behavior: Investigating Peer Influence in LLM-based Multi-Agent Systems27 May 2025
- 2025Monocle: Hybrid Local-Global In-Context Evaluation for Long-Text Generation with Uncertainty-Based Active Learning26 May 2025
38 more
- 2025Grammars of Formal Uncertainty: When to Trust LLMs in Automated Reasoning Tasks26 May 2025
- 2025Comparing Uncertainty Measurement and Mitigation Methods for Large Language Models: A Systematic Review25 Apr 2025
- 2025Your Model is Overconfident, and Other Lies We Tell Ourselves3 Mar 2025
- 2025Trust Me, I'm Wrong: LLMs Hallucinate with Certainty Despite Knowing the Answer18 Feb 2025
- 2025Uncertainty Quantification and Decomposition for LLM-based Recommendation29 Jan 2025
- 2025AdaptiveLog: An Adaptive Log Analysis Framework with the Collaboration of Large and Small Language Model19 Jan 2025
- 2024Enhancing Trust in Large Language Models via Uncertainty-Calibrated Fine-Tuning3 Dec 2024
- 2024Addressing Uncertainty in LLMs to Enhance Reliability in Generative AI4 Nov 2024
- 2024A Survey of Uncertainty Estimation in LLMs: Theory Meets Practice20 Oct 2024
- 2024Uncertainty-aware Human Mobility Modeling and Anomaly Detection2 Oct 2024
- 2024Domain-specific or Uncertainty-aware models: Does it really make a difference for biomedical text classification?17 Jul 2024
- 2024Uncertainty is Fragile: Manipulating Uncertainty in Large Language Models15 Jul 2024
- 2024Are Data Augmentation Methods in Named Entity Recognition Applicable for Uncertainty Estimation?2 Jul 2024
- 2024Uncertainty Quantification in Table Structure Recognition1 Jul 2024
- 2024Kernel Language Entropy: Fine-grained Uncertainty Quantification for LLMs from Semantic Similarities30 May 2024
- 2024Conformal Prediction for Natural Language Processing: A Survey3 May 2024
- 2024Uncertainty-boosted Robust Video Activity Anticipation29 Apr 2024
- 2024From Robustness to Improved Generalization and Calibration in Pre-trained Language Models31 Mar 2024
- 2024Self-Evaluation of Large Language Model based on Glass-box Features7 Mar 2024
- 2024Probabilistically Correct Language-based Multi-Robot Planning using Conformal Prediction23 Feb 2024
- 2024Beyond Simple Averaging: Improving NLP Ensemble Performance with Topological-Data-Analysis-Based Weighting22 Feb 2024
- 2024MARS: Meaning-Aware Response Scoring for Uncertainty Estimation in Generative LLMs19 Feb 2024
- 2024Uncertainty Quantification for In-Context Learning of Large Language Models15 Feb 2024
- 2024Consistent Joint Decision-Making with Heterogeneous Learning Models6 Feb 2024
- 2024Taking Action Towards Graceful Interaction: The Effects of Performing Actions on Modelling Policies for Instruction Clarification Requests30 Jan 2024
- 2023Active Instruction Tuning: Improving Cross-Task Generalization by Training on Prompt Sensitive Tasks1 Nov 2023
- 2023Software Entity Recognition with Noise-Robust Learning21 Aug 2023
- 2023Prompt2Gaussia: Uncertain Prompt-learning for Script Event Prediction4 Aug 2023
- 2023Robots That Ask For Help: Uncertainty Alignment for Large Language Model Planners4 Jul 2023
- 2023Hyperbolic Active Learning for Semantic Segmentation under Domain Shift19 Jun 2023
- 2023CLARA: Classifying and Disambiguating User Commands for Reliable Interactive Robotic Agents17 Jun 2023
- 2023A Bayesian approach to uncertainty in word embedding bias estimation15 Jun 2023
- 2023Efficient Uncertainty Quantification and Reduction for Over-Parameterized Neural Networks9 Jun 2023
- 2023CAVEN: An Embodied Conversational Agent for Efficient Audio-Visual Navigation in Noisy Environments6 Jun 2023
- 2023CUE: An Uncertainty Interpretation Framework for Text Classifiers Built on Pre-Trained Language Models6 Jun 2023
- 2023Uncertainty in Natural Language Processing: Sources, Quantification, and Applications5 Jun 2023
- 2023Uncertainty-Aware Bootstrap Learning for Joint Extraction on Distantly-Supervised Data5 May 2023
- 2023Reliable Gradient-free and Likelihood-free Prompt Tuning30 Apr 2023
50 newest of 79 All 79 (JSON)
- 2021On Hallucination and Predictive Uncertainty in Conditional Language Generationcited by 176
- 2024Kernel Language Entropy: Fine-grained Uncertainty Quantification for LLMs from Semantic Similaritiescited by 156
- 2024A Survey of Uncertainty Estimation in LLMs: Theory Meets Practicecited by 54
- 2023Active Instruction Tuning: Improving Cross-Task Generalization by Training on Prompt Sensitive Taskscited by 53
- 2024Conformal Prediction for Natural Language Processing: A Surveycited by 52
- 2024Uncertainty Quantification for In-Context Learning of Large Language Modelscited by 45
- 2024MARS: Meaning-Aware Response Scoring for Uncertainty Estimation in Generative LLMscited by 45
- 2023Uncertainty in Natural Language Processing: Sources, Quantification, and Applicationscited by 44
- 2022Scalable Uncertainty Quantification for Deep Operator Networks using Randomized Priorscited by 39
- 2022Weakly Supervised Data Augmentation Through Prompting for Dialogue Understandingcited by 32
- 2021Distributed NLI: Learning to Predict Human Opinion Distributions for Language Reasoningcited by 31
- 2025Trust Me, I'm Wrong: LLMs Hallucinate with Certainty Despite Knowing the Answercited by 26
12 most cited of 74 All 74 (JSON)
- 2026Credal Concept Bottleneck Models for Epistemic-Aleatoric Uncertainty Decomposition27 Apr 2026
- 2026Uncertainty and Fairness Awareness in LLM-Based Recommendation Systems31 Jan 2026
- 2025Don't Throw Away Your Beams: Improving Consistency-based Uncertainties in LLMs via Beam Search10 Dec 2025
- 2025UncertaintyZoo: A Unified Toolkit for Quantifying Predictive Uncertainty in Deep Learning Systems6 Dec 2025
- 2025HACK: Hallucinations Along Certainty and Knowledge Axes28 Oct 2025
- 2025Multi-Modal Fact-Verification Framework for Reducing Hallucinations in Large Language Models26 Oct 2025
- 2025Confidence Calibration in Large Language Model-Based Entity Matching23 Sep 2025
- 2025Are you sure? Measuring models bias in content moderation through uncertainty21 Sep 2025
- 2025Non-exchangeable Conformal Prediction for Temporal Graph Neural Networks2 Jul 2025
- 2025The Consistency Hypothesis in Uncertainty Quantification for Large Language Models27 Jun 2025
- 2025Monocle: Hybrid Local-Global In-Context Evaluation for Long-Text Generation with Uncertainty-Based Active Learning26 May 2025
- 2025Comparing Uncertainty Measurement and Mitigation Methods for Large Language Models: A Systematic Review25 Apr 2025
12 newest of 74 All 74 (JSON)