mlsanityData to 14 Oct 2026

Characterizing the Role of Similarity in the Property Inferences of Language Models

arXiv 2410.22590 ↗First posted 29 Oct 2024cs.CLcited by 11

Not on the map. Why?

Abstract (arXiv)

Property inheritance -- a phenomenon where novel properties are projected from higher level categories (e.g., birds) to lower level ones (e.g., sparrows) -- provides a unique window into how humans organize and deploy conceptual knowledge. It is debated whether this ability arises due to explicitly stored taxonomic knowledge vs. simple computations of similarity between mental representations. How are these mechanistic hypotheses manifested in contemporary language models? In this work, we investigate how LMs perform property inheritance with behavioral and causal representational analysis experiments. We find that taxonomy and categorical similarities are not mutually exclusive in LMs' property inheritance behavior. That is, LMs are more likely to project novel properties from one category to the other when they are taxonomically related and at the same time, highly similar. Our findings provide insight into the conceptual structure of language models and may suggest new psycholinguistic experiments for human subjects.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 28 of its 79 references that are in the map. First posted on arXiv 29 Oct 2024.

Builds on

28 of 79 references.

References · 28

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
  3. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  4. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
  5. 2019Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inferencecited by 876
  6. 2023Progress measures for grokking via mechanistic interpretabilitycited by 839
  7. 2019Designing and Interpreting Probes with Control Taskscited by 640
  8. 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 435
  9. 2023Towards Revealing the Mystery behind Chain of Thought: A Theoretical Perspectivecited by 294
  10. 2023Evaluating the Ripple Effects of Knowledge Editing in Language Modelscited by 242
  11. 2023Finding Alignments Between Interpretable Causal Variables and Distributed Neural Representationscited by 209
  12. 2023Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretabilitycited by 205
16 more
  1. 2021Causal Analysis of Syntactic Agreement Mechanisms in Neural Language Modelscited by 131
  2. 2021Can Language Models Encode Perceptual Structure Without Grounding? A Case Study in Colorcited by 119
  3. 2023How do Language Models Bind Entities in Context?cited by 108
  4. 2024RAVEL: Evaluating Interpretability Methods on Disentangling Language Model Representationscited by 79
  5. 2022minicons: Enabling Flexible Behavioral and Representational Analyses of Transformer Language Modelscited by 58
  6. 2024pyvene: A Library for Understanding and Improving PyTorch Models via Interventionscited by 54
  7. 2020Word meaning in minds and machinescited by 51
  8. 2023Measuring Inductive Biases of In-Context Learning with Underspecified Demonstrationscited by 49
  9. 2024CausalGym: Benchmarking causal interpretability methods on linguistic taskscited by 46
  10. 2021Do language models learn typicality judgments from text?cited by 32
  11. 2023In-context Learning Generalizes, But Not Always Robustly: The Case of Syntaxcited by 22
  12. 2019Revealing interpretable object representations from human behaviorcited by 17
  13. 2021LMMS Reloaded: Transformer-based Sense Embeddings for Disambiguation and Beyondcited by 17
  14. 2023Do PLMs Know and Understand Ontological Knowledge?cited by 13
  15. 2024TAXI: Evaluating Categorical Knowledge Editing for Language Modelscited by 7
  16. 2024Experimental Contexts Can Facilitate Robust Semantic Property Inference in Language Models, but Inconsistentlycited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 6.

11 papers.

  1. 2026Manifold Steering Reveals the Shared Geometry of Neural Network Representation and Behaviorcited by 26
  2. 2025Causal Interventions Reveal Shared Structure Across English Filler-Gap Constructionscited by 21
  3. 2025How Causal Abstraction Underpins Computational Explanationcited by 11
  4. 2026Arithmetic in the Wild: Llama uses Base-10 Addition to Reason About Cyclic Conceptscited by 10
  5. 2025Vision-and-Language Training Helps Deploy Taxonomic Knowledge but Does Not Fundamentally Alter Itcited by 6
  6. 2025RankAlign: A Ranking View of the Generator-Validator Gap in Large Language Modelscited by 4
  7. 2025WUGNECTIVES: Novel Entity Inferences of Language Models from Discourse Connectivescited by 2
  8. 2026Cross-Modal Taxonomic Generalization in (Vision-) Language Modelscited by 1
  9. 2026Bears, all bears, and some bears. Language Constraints on Language Models' Inductive Inferencescited by 0
  10. 2026Multimodal QUD: Inquisitive Questions from Scientific Figurescited by 0
  11. 2026Latent Taxonomies: How Property Induction Develops in Language Model Pre-trainingcited by 0