mlsanityData to 14 Oct 2026

Reasoning about Ambiguous Definite Descriptions

arXiv 2310.14657 ↗First posted 23 Oct 2023cs.CL cs.AIcited by 0

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Natural language reasoning plays an increasingly important role in improving language models' ability to solve complex language understanding tasks. An interesting use case for reasoning is the resolution of context-dependent ambiguity. But no resources exist to evaluate how well Large Language Models can use explicit reasoning to resolve ambiguity in language. We propose to use ambiguous definite descriptions for this purpose and create and publish the first benchmark dataset consisting of such phrases. Our method includes all information required to resolve the ambiguity in the prompt, which means a model does not require anything but reasoning to do well. We find this to be a challenging task for recent LLMs. Code and data available at: https://github.com/sfschouten/exploiting-ambiguity

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 9 of its 11 references that are in the map. First posted on arXiv 23 Oct 2023.

Builds on

9 of 11 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023Logical Reasoning over Natural Language as Knowledge Representation: A Survey21 Mar
  2. 2023LLaMA: Open and Efficient Foundation Language Models27 Feb

Other references · 7

  1. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,807
  2. 2023OpenAssistant Conversations -- Democratizing Large Language Model Alignmentcited by 721
  3. 2020AmbigQA: Answering Ambiguous Open-domain Questionscited by 390
  4. 2022Reasoning with Language Model Prompting: A Surveycited by 252
  5. 2023We're Afraid Language Models Aren't Modeling Ambiguitycited by 126
  6. 2022The Defeat of the Winograd Schema Challengecited by 25
  7. 2023AmbiCoref: Evaluating Human and Model Sensitivity to Ambiguous Coreferencecited by 12

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Small Language Model Helps Resolve Semantic Ambiguity of LLM Prompt
  2. 2022Towards Reasoning in Large Language Models: A Survey
  3. 2025Negation: A Pink Elephant in the Large Language Models' Room?
  4. 2025It Depends: Resolving Referential Ambiguity in Minimal Contexts with Commonsense Knowledge
  5. 2025Reasoning Capabilities and Invariability of Large Language Models
  6. 2025On the Notion that Language Models Reason
  7. 2026From Answers to Interpretations: Rethinking Ambiguity-Induced Aleatoric Uncertainty Estimation in LLMs
  8. 2023Simple Linguistic Inferences of Large Language Models (LLMs): Blind Spots and Blinds