mlsanityData to 7 Oct 2026

Uncertainty-Aware Natural Language Inference with Stochastic Weight Averaging

arXiv 2304.04726 ↗First posted 10 Apr 2023cs.CLcited by 4

Not on the map. Why?

Abstract (arXiv)

This paper introduces Bayesian uncertainty modeling using Stochastic Weight Averaging-Gaussian (SWAG) in Natural Language Understanding (NLU) tasks. We apply the approach to standard tasks in natural language inference (NLI) and demonstrate the effectiveness of the method in terms of prediction accuracy and correlation with human annotation disagreements. We argue that the uncertainty representations in SWAG better reflect subjective interpretation and the natural variation that is also present in human language understanding. The results reveal the importance of uncertainty modeling, an often neglected aspect of neural language modeling, in NLU tasks.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 17 of its 24 references that are in the map. First posted on arXiv 10 Apr 2023.

Builds on

17 of 24 references.

References · 17

  1. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  2. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  3. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  4. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  5. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  6. 2018Averaging Weights Leads to Wider Optima and Better Generalizationcited by 1,194
  7. 2019A Simple Baseline for Bayesian Uncertainty in Deep Learningcited by 555
  8. 2020Bayesian Deep Learning and a Probabilistic Perspective of Generalizationcited by 480
  9. 2019Human uncertainty makes classification more robustcited by 231
  10. 2020What Can We Learn from Collective Human Opinions on Natural Language Inference Data?cited by 148
  11. 2022The 'Problem' of Human Label Variation: On Ground Truth in Data, Modeling and Evaluationcited by 133
  12. 2022When Do Flat Minima Optimizers Work?cited by 78
5 more
  1. 2019Uncertain Natural Language Inferencecited by 35
  2. 2018Testing the Generalization Power of Neural Network Models Across NLI Benchmarkscited by 15
  3. 2022Revisiting Checkpoint Averaging for Neural Machine Translationcited by 13
  4. 2022Improving Generalization of Pre-trained Language Models via Stochastic Weight Averagingcited by 9
  5. 2021How Emotionally Stable is ALBERT? Testing Robustness with Stochastic Weight Averaging on a Sentiment Analysis Taskcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 1; 2025: 1; 2026: 1.

4 papers.

  1. 2023Shifting Attention to Relevance: Towards the Predictive Uncertainty Quantification of Free-Form Large Language Modelscited by 147
  2. 2025Robust Hallucination Detection in LLMs via Adaptive Token Selectioncited by 19
  3. 2026UAT-LITE: Inference-Time Uncertainty-Aware Attention for Pretrained Transformerscited by 4
  4. 2024Defeasible Visual Entailment: Benchmark, Evaluator, and Reward-Driven Optimizationcited by 3