mlsanityData to 7 Oct 2026

AmbiCoref: Evaluating Human and Model Sensitivity to Ambiguous Coreference

arXiv 2302.00762 ↗First posted 1 Feb 2023cs.CLcited by 12

Not on the map. Why?

Abstract (arXiv)

Given a sentence "Abby told Brittney that she upset Courtney", one would struggle to understand who "she" refers to, and ask for clarification. However, if the word "upset" were replaced with "hugged", "she" unambiguously refers to Abby. We study if modern coreference resolution models are sensitive to such pronominal ambiguity. To this end, we construct AmbiCoref, a diagnostic corpus of minimal sentence pairs with ambiguous and unambiguous referents. Our examples generalize psycholinguistic studies of human perception of ambiguity around particular arrangements of verbs and their arguments. Analysis shows that (1) humans are less sure of referents in ambiguous AmbiCoref examples than unambiguous ones, and (2) most coreference models show little difference in output between ambiguous and unambiguous pairs. We release AmbiCoref as a diagnostic corpus for testing whether models treat ambiguity similarly to humans.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 9 of its 30 references that are in the map. First posted on arXiv 1 Feb 2023.

Builds on

9 of 30 references.

References · 9

  1. 2019SpanBERT: Improving Pre-training by Representing and Predicting Spanscited by 832
  2. 2018Gender Bias in Coreference Resolution: Evaluation and Debiasing Methodscited by 657
  3. 2020How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answeringcited by 439
  4. 2020AmbigQA: Answering Ambiguous Open-domain Questionscited by 385
  5. 2020Calibration of Pre-trained Transformerscited by 307
  6. 2019BERT for Coreference Resolution: Baselines and Analysiscited by 125
  7. 2016Deep Reinforcement Learning for Mention-Ranking Coreference Modelscited by 63
  8. 2019A Crowdsourced Frame Disambiguation Corpus with Ambiguitycited by 16
  9. 2021The Language Model Understood the Prompt was Ambiguous: Probing Syntactic Uncertainty Through Generationcited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 5; 2025: 2; 2026: 1.

12 papers.

  1. 2023We're Afraid Language Models Aren't Modeling Ambiguitycited by 124
  2. 2023Clarify When Necessary: Resolving Ambiguity Through Interaction with LMscited by 93
  3. 2024Modeling Future Conversation Turns to Teach LLMs to Ask Clarifying Questionscited by 70
  4. 2024Aligning Language Models to Explicitly Handle Ambiguitycited by 55
  5. 2024CLAMBER: A Benchmark of Identifying and Clarifying Ambiguous Information Needs in Large Language Modelscited by 48
  6. 2024AMBROSIA: A Benchmark for Parsing Ambiguous Questions into Database Queriescited by 32
  7. 2023Zero and Few-shot Semantic Parsing with Ambiguous Inputscited by 16
  8. 2024A Taxonomy of Ambiguity Types for NLPcited by 7
  9. 2025Who Relies More on World Knowledge and Bias for Syntactic Ambiguity Resolution: Humans or LLMs?cited by 3
  10. 2025Correct-Detect: Balancing Performance and Ambiguity Through the Lens of Coreference Resolution in LLMscited by 1
  11. 2026Generative AI for Managerial Decision-Making under Ambiguity and Sycophancycited by 1
  12. 2023Reasoning about Ambiguous Definite Descriptionscited by 0