mlsanityData to 7 Oct 2026

Geometric Signatures of Compositionality Across a Language Model's Lifetime

arXiv 2410.01444 ↗First posted 2 Oct 2024cs.CL cs.AI cs.ITcited by 14

Not on the map. Why?

Abstract (arXiv)

By virtue of linguistic compositionality, few syntactic rules and a finite lexicon can generate an unbounded number of sentences. That is, language, though seemingly high-dimensional, can be explained using relatively few degrees of freedom. An open question is whether contemporary language models (LMs) reflect the intrinsic simplicity of language that is enabled by compositionality. We take a geometric view of this problem by relating the degree of compositionality in a dataset to the intrinsic dimension (ID) of its representations under an LM, a measure of feature complexity. We find not only that the degree of dataset compositionality is reflected in representations' ID, but that the relationship between compositionality and geometric complexity arises due to learned linguistic features over training. Finally, our analyses reveal a striking contrast between nonlinear and linear dimensionality, showing they respectively encode semantic and superficial aspects of linguistic composition.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 49 of its 105 references that are in the map. First posted on arXiv 2 Oct 2024.

Builds on

49 of 105 references.

References · 49

  1. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  2. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
  3. 2022Emergent Abilities of Large Language Modelscited by 2,483
  4. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  5. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
  6. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,054
  7. 2016The LAMBADA dataset: Word prediction requiring a broad discourse contextcited by 960
  8. 2017Crowdsourcing Multiple Choice Science Questionscited by 742
  9. 2020Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuningcited by 658
  10. 2023Dissecting Recall of Factual Associations in Auto-Regressive Language Modelscited by 512
  11. 2018Measuring the Intrinsic Dimension of Objective Landscapescited by 364
  12. 2021The Intrinsic Dimension of Images and Its Impact on Learningcited by 307
37 more
  1. 2019Intrinsic dimension of data representations in deep neural networkscited by 276
  2. 2019Compositionality decomposed: how do neural networks generalise?cited by 264
  3. 2018Estimating the intrinsic dimension of datasets by a minimal neighborhood informationcited by 246
  4. 2020Retrieve, Program, Repeat: Complex Knowledge Base Question Answering via Alternate Meta-learningcited by 216
  5. 2024Not All Language Model Features Are One-Dimensionally Linearcited by 201
  6. 2024The Geometry of Categorical and Hierarchical Concepts in Large Language Modelscited by 151
  7. 2021All Bark and No Bite: Rogue Dimensions in Transformer Language Models Obscure Representational Qualitycited by 150
  8. 2023Sudden Drops in the Loss: Syntax Acquisition, Phase Transitions, and Simplicity Bias in MLMscited by 121
  9. 2023Intrinsic Dimension Estimation for Robust Detection of AI-Generated Textscited by 111
  10. 2023The geometry of hidden representations of large transformer modelscited by 108
  11. 2020Compositionality and Generalization in Emergent Languagescited by 103
  12. 2019Measuring Compositionality in Representation Learningcited by 84
  13. 2023The Transient Nature of Emergent In-Context Learning in Transformerscited by 72
  14. 2017Classification and Geometry of General Perceptual Manifoldscited by 62
  15. 2022Understanding Transformer Memorization Recall Through Idiomscited by 62
  16. 2024LLM Circuit Analyses Are Consistent Across Training and Scalecited by 60
  17. 2019Linguistic generalization and compositionality in modern artificial neural networkscited by 58
  18. 2024Emergence of a High-Dimensional Abstraction Phase in Language Transformerscited by 56
  19. 2022Outliers Dimensions that Disrupt Transformers Are Driven by Frequencycited by 55
  20. 2023Transformers learn through gradual rank increasecited by 52
  21. 2022Characterizing Intrinsic Compositionality in Transformers with Tree Projectionscited by 40
  22. 2020Emergence of Separable Manifolds in Deep Language Representationscited by 39
  23. 2021The Low-Dimensional Linear Geometry of Contextualized Word Representationscited by 39
  24. 2019Dimensionality compression and expansion in Deep Neural Networkscited by 38
  25. 2022Can Transformer be Too Compositional? Analysing Idiom Processing in Neural Machine Translationcited by 28
  26. 2023Bridging Information-Theoretic and Geometric Compression in Language Modelscited by 26
  27. 2024A Percolation Model of Emergence: Analyzing Transformers Trained on a Formal Languagecited by 26
  28. 2021Interpreting neural computations by examining intrinsic and embedding dimensionality of neural activitycited by 24
  29. 2016Neural versus Phrase-Based Machine Translation Quality: a Case Studycited by 19
  30. 2019Estimating the effective dimension of large biological datasets using Fisher separability analysiscited by 19
  31. 2023Fine-tuning Happens in Tiny Subspaces: Exploring Intrinsic Task-specific Subspaces of Pre-trained Language Modelscited by 19
  32. 2023Outlier Dimensions Encode Task-Specific Knowledgecited by 15
  33. 2021Syntactic Perturbations Reveal Representational Correlates of Hierarchical Phrase Structure in Pretrained Language Modelscited by 14
  34. 2024The representation landscape of few-shot learning and fine-tuning in large language modelscited by 14
  35. 2023Representation Learning via Manifold Flattening and Reconstructioncited by 11
  36. 2023Characterizing Large Language Model Geometry Helps Solve Toxicity Detection and Generationcited by 11
  37. 2024Interpretability of Language Models via Task Spacescited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 7; 2026: 6.

14 papers.

Sort:

By area:

  1. 2026From Directions to Regions: Decomposing Activations in Language Models via Local Geometrycited by 10
  2. 2025Distinct Computations Emerge From Compositional Curricula in In-Context Learningcited by 5
  3. 2025From Topology to Retrieval: Decoding Embedding Spaces with Unified Signaturescited by 4
  4. 2025TRACE for Tracking the Emergence of Semantic Representations in Transformerscited by 3
  5. 2026Tracing the complexity profiles of different linguistic phenomena through the intrinsic dimension of LLM representationscited by 3
  6. 2025Semantic Communication meets System 2 ML: How Abstraction, Compositionality and Emergent Languages Shape Intelligencecited by 2
  7. 2025Emergent Convergence in Multi-Agent LLM Annotationcited by 2
  8. 2026Abstraction Induces the Brain Alignment of Language and Speech Modelscited by 2
  9. 2026Topological Signatures of Grokkingcited by 2
  10. 2024A Comparative Study of Learning Paradigms in Large Language Models via Intrinsic Dimensioncited by 1
  11. 2025Unveiling Intrinsic Dimension of Texts: from Academic Abstract to Creative Storycited by 1
  12. 2025Token Homogenization under Positional Biascited by 0
2 more
  1. 2026surprisal is Not a Theorycited by 0
  2. 2026Representation Measurements Under Function-Preserving Reparameterizationscited by 0