mlsanityData to 7 Oct 2026

WUGNECTIVES: Novel Entity Inferences of Language Models from Discourse Connectives

arXiv 2510.09556 ↗First posted 10 Oct 2025cs.CLcited by 1

Not on the map. Why?

Abstract (arXiv)

The role of world knowledge has been particularly crucial to predict the discourse connective that marks the discourse relation between two arguments, with language models (LMs) being generally successful at this task. We flip this premise in our work, and instead study the inverse problem of understanding whether discourse connectives can inform LMs about the world. To this end, we present WUGNECTIVES, a dataset of 8,880 stimuli that evaluates LMs' inferences about novel entities in contexts where connectives link the entities to particular attributes. On investigating 17 different LMs at various scales, and training regimens, we found that tuning an LM to show reasoning behavior yields noteworthy improvements on most connectives. At the same time, there was a large variation in LMs' overall performance across connective type, with all models systematically struggling on connectives that express a concessive meaning. Our findings pave the way for more nuanced investigations into the functional role of language cues as captured by LMs. We release WUGNECTIVES at https://github.com/kanishkamisra/wugnectives

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 14 of its 62 references that are in the map. First posted on arXiv 10 Oct 2025.

Builds on

14 of 62 references.

References · 14

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2024Qwen2.5 Technical Reportcited by 4,713
  3. 2024Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Researchcited by 475
  4. 2021Causal Abstractions of Neural Networkscited by 401
  5. 2021Can Language Models Encode Perceptual Structure Without Grounding? A Case Study in Colorcited by 118
  6. 2019ASER: A Large-scale Eventuality Knowledge Graphcited by 89
  7. 2022minicons: Enabling Flexible Behavioral and Representational Analyses of Transformer Language Modelscited by 58
  8. 2024Elements of World Knowledge (EWoK): A Cognition-Inspired Framework for Evaluating Basic World Knowledge in Language Modelscited by 55
  9. 2025How Linguistics Learned to Stop Worrying and Love the Language Modelscited by 50
  10. 2021Discourse Probing of Pretrained Language Modelscited by 38
  11. 2021Pragmatic competence of pre-trained language models through the lens of discourse connectivescited by 16
  12. 2021Is Incoherence Surprising? Targeted Evaluation of Coherence Prediction from Language Modelscited by 15
2 more
  1. 2022WinoDict: Probing language models for in-context word acquisitioncited by 14
  2. 2024Characterizing the Role of Similarity in the Property Inferences of Language Modelscited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Would you still call this Dax? Novel Visual References in VLMs and Humanscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025PragWorld: A Benchmark Evaluating LLMs' Local World Model under Minimal Linguistic Alterations and Conversational Dynamics
  2. 2024Monitoring Latent World States in Language Models with Propositional Probes
  3. 2023Simple Linguistic Inferences of Large Language Models (LLMs): Blind Spots and Blinds
  4. 2023WorldSense: A Synthetic Benchmark for Grounded Reasoning in Large Language Models
  5. 2023Large Linguistic Models: Investigating LLMs' metalinguistic abilities
  6. 2023From Word Models to World Models: Translating from Natural Language to the Probabilistic Language of Thought
  7. 2024Calibrating Reasoning in Language Models with Internal Consistency
  8. 2026From Concept Alignment to Causal Grounding: An Intervention Test of Chain-of-Thought Faithfulness