mlsanityData to 14 Oct 2026

(V)LMs generalize beyond surface co-occurrence: Evidence from cross-modal number agreement

arXiv 2609.00443 ↗First posted 31 Aug 2026cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Language models learn about grammatical number primarily from co-occurrence, and show frequency effects as a result---sometimes taken to indicate that they do not learn abstract ``rules'', and are instead dependent on specific lexical items. Testing generalization with text stimuli alone cannot settle this debate, since distributional cues (is/are, this/these) easily give number away. We instead use cross-modal generalization as a tool to investigate abstractions in LMs that can also accept visual inputs (VLMs), restricting the evidence that diagnoses number to an extra-linguistic modality. We teach VLMs pairs of new nouns by adding new embeddings and only updating them during learning, comparing conditions where number is diagnosed by visual cues alone against ones where it is disambiguated by text. Across behavior, representational dynamics, and causal mechanisms, we find non-trivial evidence for cross-modal generalization across both exposure conditions, and that linguistic vs. extra-linguistic cue conditions are treated in similar ways in the internal mechanisms of the model. This suggests that statistical learners like VLMs can generalize beyond surface-level co-occurrence and show genuine abstraction-compatible behavior.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 27 of its 54 references that are in the map. First posted on arXiv 31 Aug 2026.

Builds on

27 of 54 references.

References · 27

  1. 2016Understanding intermediate layers using linear classifier probescited by 1,353
  2. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,312
  3. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 712
  4. 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 435
  5. 2021Causal Abstractions of Neural Networkscited by 422
  6. 2016Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependenciescited by 354
  7. 2018Targeted Syntactic Evaluation of Language Modelscited by 242
  8. 2018Colorless green recurrent networks dream hierarchicallycited by 213
  9. 2023Finding Alignments Between Interpretable Causal Variables and Distributed Neural Representationscited by 209
  10. 2020A Systematic Assessment of Syntactic Generalization in Neural Language Modelscited by 165
  11. 2021Causal Analysis of Syntactic Agreement Mechanisms in Neural Language Modelscited by 131
  12. 2019The emergence of number and syntax units in LSTM language modelscited by 94
15 more
  1. 2021Frequency Effects on Syntactic Rule Learning in Transformerscited by 59
  2. 2022Probing for the Usage of Grammatical Numbercited by 59
  3. 2022minicons: Enabling Flexible Behavioral and Representational Analyses of Transformer Language Modelscited by 58
  4. 2020Cross-Linguistic Syntactic Evaluation of Word Prediction Modelscited by 55
  5. 2024pyvene: A Library for Understanding and Improving PyTorch Models via Interventionscited by 54
  6. 2024CausalGym: Benchmarking causal interpretability methods on linguistic taskscited by 46
  7. 2025We Can't Understand AI Using our Existing Vocabularycited by 25
  8. 2022Does BERT really agree ? Fine-grained Analysis of Lexical Dependence on a Syntactic Taskcited by 13
  9. 2023How Abstract Is Linguistic Generalization in Large Language Models? Experiments with Argument Structurecited by 13
  10. 2017One-shot and few-shot learning of word embeddingscited by 7
  11. 2022Do Language Models Learn Position-Role Mappings?cited by 5
  12. 2023Abstraction via exemplars? A representational case study on lexical category inference in BERTcited by 5
  13. 2025Rapid Word Learning Through Meta In-Context Learningcited by 4
  14. 2026Humans and transformer LMs: Abstraction drives language learningcited by 2
  15. 2026Cross-Modal Taxonomic Generalization in (Vision-) Language Modelscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025The in-context inductive biases of vision-language models differ across modalities
  2. 2025Too Big to Think: Capacity, Memorization, and Generalization in Pre-Trained Transformers
  3. 2025Quantification and object perception in Multimodal Large Language Models and human linguistic cognition
  4. 2025Non-literal Understanding of Number Words by Language Models
  5. 2025Toward a Theory of Generalizability in LLM Mechanistic Interpretability Research
  6. 2025Is This Just Fantasy? Language Model Representations Reflect Human Judgments of Event Plausibility
  7. 2026Beyond Hooking Onto the World: Referential Profiles and the Numerical Structure of LLM Grounding
  8. 2025Understanding Counting Mechanisms in Large Language and Vision-Language Models