# From Disagreement to Understanding: The Case for Ambiguity Detection in NLI

arXiv: https://arxiv.org/abs/2507.15114 · HTML: https://mlsanity.com/paper/2507.15114/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 24 references that are in the map.
- First posted on arXiv 20 Jul 2025.

## Builds on (most cited first, 13 of 13)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [A large annotated corpus for learning natural language inference](https://mlsanity.com/paper/1508.05326/) (2015)
- [FEVER: a large-scale dataset for Fact Extraction and VERification](https://mlsanity.com/paper/1803.05355/) (2018)
- [What Can We Learn from Collective Human Opinions on Natural Language Inference Data?](https://mlsanity.com/paper/2010.03532/) (2020)
- [We're Afraid Language Models Aren't Modeling Ambiguity](https://mlsanity.com/paper/2304.14399/) (2023)
- [VariErr NLI: Separating Annotation Error from Human Label Variation](https://mlsanity.com/paper/2403.01931/) (2024)
- [Distributed NLI: Learning to Predict Human Opinion Distributions for Language Reasoning](https://mlsanity.com/paper/2104.08676/) (2021)
- [Embracing Ambiguity: Shifting the Training Target of NLI Models](https://mlsanity.com/paper/2106.03020/) (2021)
- [Ecologically Valid Explanations for Label Variation in NLI](https://mlsanity.com/paper/2310.13850/) (2023)
- [A Taxonomy of Ambiguity Types for NLP](https://mlsanity.com/paper/2403.14072/) (2024)
- [Discourse Marker Augmented Network with Reinforcement Learning for Natural Language Inference](https://mlsanity.com/paper/1907.09692/) (2019)
- [AMREx: AMR for Explainable Fact Verification](https://mlsanity.com/paper/2411.01343/) (2024)

## Built on by (most cited first, 4 of 4)

- [SHALA-LLM: Smartly Handling Ambiguous Labels in Aligning LLMs](https://mlsanity.com/paper/2606.05376/) (2026)
- [How Much Human Label Variation Does Formal Semantic Structure Explain? Group-Level Effects and Item-Level Ceilings in NLI](https://mlsanity.com/paper/2607.15870/) (2026)
- [EVADE: LLM-Based Explanation Generation and Validation for Error Detection in NLI](https://mlsanity.com/paper/2511.08949/) (2025)
- [Whose Ground Truth? Embracing Ambiguity in Human-Centered AI](https://mlsanity.com/paper/2610.10805/) (2026)

## Abstract (arXiv)

This position paper argues that annotation disagreement in Natural Language Inference (NLI) is not mere noise but often reflects meaningful variation, especially when triggered by ambiguity in the premise or hypothesis. While underspecified guidelines and annotator behavior contribute to variation, content-based ambiguity provides a process-independent signal of divergent human perspectives. We call for a shift toward ambiguity-aware NLI that first identifies ambiguous input pairs, classifies their types, and only then proceeds to inference. To support this shift, we present a framework that incorporates ambiguity detection and classification prior to inference. We also introduce a unified taxonomy that synthesizes existing taxonomies, illustrates key subtypes with examples, and motivates targeted detection methods that better align models with human interpretation. Although current resources lack datasets explicitly annotated for ambiguity and subtypes, this gap presents an opportunity: by developing new annotated resources and exploring unsupervised approaches to ambiguity detection, we enable more robust, explainable, and human-aligned NLI systems.