mlsanityData to 7 Oct 2026

Do Language Models Know Theo Has a Wife? Investigating the Proviso Problem

arXiv 2603.08358 ↗First posted 9 Mar 2026cs.CLcited by 1

Not on the map. Why?

Abstract (arXiv)

We investigate how language models handle the proviso problem, an unresolved issue in pragmatics where presuppositions in conditional sentences diverge between theoretical and human interpretations. We reformulate this phenomenon as a Natural Language Inference task and introduce a diagnostic dataset designed to probe presupposition projection in conditionals. We evaluate RoBERTa, DeBERTa, LLaMA, and Gemma using explainability analyses. The results show that models broadly align with human judgments but rely on shallow pattern matching rather than semantic or pragmatic reasoning. Our work provides the first computational evaluation framework for the proviso problem and highlights the need for diagnostic, multi-method approaches to assess pragmatic competence and context-dependent meaning in language models.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 12 of its 42 references that are in the map. First posted on arXiv 9 Mar 2026.

Builds on

12 of 42 references.

References · 12

  1. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  2. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  3. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  4. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  5. 2016ConceptNet 5.5: An Open Multilingual Graph of General Knowledgecited by 1,065
  6. 2018SWAG: A Large-Scale Adversarial Dataset for Grounded Commonsense Inferencecited by 333
  7. 2020A Diagnostic Study of Explainability Techniques for Text Classificationcited by 158
  8. 2020The elephant in the interpretability room: Why use attention as explanation when we have saliency methods?cited by 122
  9. 2020Are Natural Language Inference Models IMPPRESsive? Learning IMPlicature and PRESuppositioncited by 76
  10. 2019Investigating BERT's Knowledge of Language: Five Analysis Methods with NPIscited by 58
  11. 2021NOPE: A Corpus of Naturally-Occurring Presuppositions in Englishcited by 23
  12. 2025Let's CONFER: A Dataset for Evaluating Natural Language Inference Models on CONditional InFERence and Presuppositioncited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.