mlsanityData to 7 Oct 2026

Uncertain Natural Language Inference

arXiv 1909.03042 ↗First posted 6 Sep 2019cs.CLcited by 35

Not on the map. Why?

Abstract (arXiv)

We introduce Uncertain Natural Language Inference (UNLI), a refinement of Natural Language Inference (NLI) that shifts away from categorical labels, targeting instead the direct prediction of subjective probability assessments. We demonstrate the feasibility of collecting annotations for UNLI by relabeling a portion of the SNLI dataset under a probabilistic scale, where items even with the same categorical label differ in how likely people judge them to be true given a premise. We describe a direct scalar regression modeling approach, and find that existing categorically labeled NLI data can be used in pre-training. Our best models approach human performance, demonstrating models may be capable of more subtle inferences than the categorical bin assignment employed in current NLI tasks.

Key facts. Cited by 35 papers in the map (data through 7 Oct 2026). First posted on arXiv 6 Sep 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 7; 2022: 6; 2023: 13; 2024: 4; 2025: 3; 2026: 2.

35 papers.

Sort:

By area:

  1. 2023We're Afraid Language Models Aren't Modeling Ambiguitycited by 124
  2. 2022Stop Measuring Calibration When Humans Disagreecited by 70
  3. 2023Natural Language Reasoning, A Surveycited by 61
  4. 2022Natural Language Deduction through Search over Statement Compositionscited by 35
  5. 2021Distributed NLI: Learning to Predict Human Opinion Distributions for Language Reasoningcited by 31
  6. 2024Beyond correlation: The Impact of Human Uncertainty in Measuring the Effectiveness of Automatic Evaluation and LLM-as-a-Judgecited by 29
  7. 2021Learning with Different Amounts of Annotation: From Zero to Many Labelscited by 28
  8. 2023MPCHAT: Towards Multimodal Persona-Grounded Conversationcited by 24
  9. 2021NOPE: A Corpus of Naturally-Occurring Presuppositions in Englishcited by 23
  10. 2021Embracing Ambiguity: Shifting the Training Target of NLI Modelscited by 22
  11. 2021Refined Commonsense Knowledge from Large-Scale Web Contentscited by 19
  12. 2024BIRD: A Trustworthy Bayesian Inference Framework for Large Language Modelscited by 18
23 more
  1. 2023Zero and Few-shot Semantic Parsing with Ambiguous Inputscited by 16
  2. 2023Design Choices for Crowdsourcing Implicit Discourse Relations: Revealing the Biases Introduced by Task Designcited by 14
  3. 2022Ambiguous Images With Human Judgments for Robust Visual Event Classificationcited by 12
  4. 2023Using Natural Language Explanations to Rescale Human Judgmentscited by 11
  5. 2021Clean or Annotate: How to Spend a Limited Data Collection Budgetcited by 10
  6. 2023MegaWika: Millions of reports and their sources across 50 diverse languagescited by 9
  7. 2025Always Tell Me The Odds: Fine-grained Conditional Probability Estimationcited by 9
  8. 2023Collective Human Opinions in Semantic Textual Similaritycited by 8
  9. 2022Probing neural language models for understanding of words of estimative probabilitycited by 7
  10. 2021Capturing Label Distribution: A Case Study in NLIcited by 6
  11. 2023Investigating Multi-source Active Learning for Natural Language Inferencecited by 6
  12. 2023Uncertainty-Aware Natural Language Inference with Stochastic Weight Averagingcited by 4
  13. 2023No Strong Feelings One Way or Another: Re-operationalizing Neutrality in Natural Language Inferencecited by 4
  14. 2022Possible Stories: Evaluating Situated Commonsense Reasoning under Multiple Possible Scenarioscited by 3
  15. 2022Pneg: Prompt-based Negative Response Generation for Dialogue Response Selection Taskcited by 3
  16. 2024Defeasible Visual Entailment: Benchmark, Evaluator, and Reward-Driven Optimizationcited by 3
  17. 2024Interpretable Multimodal Out-of-context Detection with Soft Logic Regularizationcited by 2
  18. 2023On Degrees of Freedom in Defining and Testing Natural Language Understandingcited by 1
  19. 2025Bonsai: Interpretable Tree-Adaptive Grounded Reasoningcited by 1
  20. 2026Humans and LLMs Diverge on Probabilistic Inferencescited by 1
  21. 2026Process Supervision of Confidence Margin for Calibrated LLM Reasoningcited by 1
  22. 2023Revisiting Acceptability Judgementscited by 0
  23. 2025Charting the Design Space of Neural Graph Representations for Subgraph Matchingcited by 0