# Geometric Signatures of Compositionality Across a Language Model's Lifetime

arXiv: https://arxiv.org/abs/2410.01444 · HTML: https://mlsanity.com/paper/2410.01444/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 49 of its 105 references that are in the map.
- First posted on arXiv 2 Oct 2024.

## Builds on (most cited first, 30 of 49)

- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [PIQA: Reasoning about Physical Commonsense in Natural Language](https://mlsanity.com/paper/1911.11641/) (2019)
- [Emergent Abilities of Large Language Models](https://mlsanity.com/paper/2206.07682/) (2022)
- [The Pile: An 800GB Dataset of Diverse Text for Language Modeling](https://mlsanity.com/paper/2101.00027/) (2020)
- [Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling](https://mlsanity.com/paper/2304.01373/) (2023)
- [Measuring and Narrowing the Compositionality Gap in Language Models](https://mlsanity.com/paper/2210.03350/) (2022)
- [The LAMBADA dataset: Word prediction requiring a broad discourse context](https://mlsanity.com/paper/1606.06031/) (2016)
- [Crowdsourcing Multiple Choice Science Questions](https://mlsanity.com/paper/1707.06209/) (2017)
- [Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning](https://mlsanity.com/paper/2012.13255/) (2020)
- [Dissecting Recall of Factual Associations in Auto-Regressive Language Models](https://mlsanity.com/paper/2304.14767/) (2023)
- [Measuring the Intrinsic Dimension of Objective Landscapes](https://mlsanity.com/paper/1804.08838/) (2018)
- [The Intrinsic Dimension of Images and Its Impact on Learning](https://mlsanity.com/paper/2104.08894/) (2021)
- [Intrinsic dimension of data representations in deep neural networks](https://mlsanity.com/paper/1905.12784/) (2019)
- [Compositionality decomposed: how do neural networks generalise?](https://mlsanity.com/paper/1908.08351/) (2019)
- [Estimating the intrinsic dimension of datasets by a minimal neighborhood information](https://mlsanity.com/paper/1803.06992/) (2018)
- [Retrieve, Program, Repeat: Complex Knowledge Base Question Answering via Alternate Meta-learning](https://mlsanity.com/paper/2010.15875/) (2020)
- [Not All Language Model Features Are One-Dimensionally Linear](https://mlsanity.com/paper/2405.14860/) (2024)
- [The Geometry of Categorical and Hierarchical Concepts in Large Language Models](https://mlsanity.com/paper/2406.01506/) (2024)
- [All Bark and No Bite: Rogue Dimensions in Transformer Language Models Obscure Representational Quality](https://mlsanity.com/paper/2109.04404/) (2021)
- [Sudden Drops in the Loss: Syntax Acquisition, Phase Transitions, and Simplicity Bias in MLMs](https://mlsanity.com/paper/2309.07311/) (2023)
- [The geometry of hidden representations of large transformer models](https://mlsanity.com/paper/2302.00294/) (2023)
- [Intrinsic Dimension Estimation for Robust Detection of AI-Generated Texts](https://mlsanity.com/paper/2306.04723/) (2023)
- [Compositionality and Generalization in Emergent Languages](https://mlsanity.com/paper/2004.09124/) (2020)
- [Measuring Compositionality in Representation Learning](https://mlsanity.com/paper/1902.07181/) (2019)
- [The Transient Nature of Emergent In-Context Learning in Transformers](https://mlsanity.com/paper/2311.08360/) (2023)
- [LLM Circuit Analyses Are Consistent Across Training and Scale](https://mlsanity.com/paper/2407.10827/) (2024)
- [Classification and Geometry of General Perceptual Manifolds](https://mlsanity.com/paper/1710.06487/) (2017)
- [Understanding Transformer Memorization Recall Through Idioms](https://mlsanity.com/paper/2210.03588/) (2022)
- [Linguistic generalization and compositionality in modern artificial neural networks](https://mlsanity.com/paper/1904.00157/) (2019)
- [Emergence of a High-Dimensional Abstraction Phase in Language Transformers](https://mlsanity.com/paper/2405.15471/) (2024)

## Built on by (most cited first, 16 of 16)

- [From Directions to Regions: Decomposing Activations in Language Models via Local Geometry](https://mlsanity.com/paper/2602.02464/) (2026)
- [Distinct Computations Emerge From Compositional Curricula in In-Context Learning](https://mlsanity.com/paper/2506.13253/) (2025)
- [From Topology to Retrieval: Decoding Embedding Spaces with Unified Signatures](https://mlsanity.com/paper/2511.22150/) (2025)
- [TRACE for Tracking the Emergence of Semantic Representations in Transformers](https://mlsanity.com/paper/2505.17998/) (2025)
- [Tracing the complexity profiles of different linguistic phenomena through the intrinsic dimension of LLM representations](https://mlsanity.com/paper/2601.03779/) (2026)
- [Semantic Communication meets System 2 ML: How Abstraction, Compositionality and Emergent Languages Shape Intelligence](https://mlsanity.com/paper/2505.20964/) (2025)
- [Emergent Convergence in Multi-Agent LLM Annotation](https://mlsanity.com/paper/2512.00047/) (2025)
- [Abstraction Induces the Brain Alignment of Language and Speech Models](https://mlsanity.com/paper/2602.04081/) (2026)
- [Topological Signatures of Grokking](https://mlsanity.com/paper/2605.06352/) (2026)
- [A Comparative Study of Learning Paradigms in Large Language Models via Intrinsic Dimension](https://mlsanity.com/paper/2412.06245/) (2024)
- [Unveiling Intrinsic Dimension of Texts: from Academic Abstract to Creative Story](https://mlsanity.com/paper/2511.15210/) (2025)
- [Token Homogenization under Positional Bias](https://mlsanity.com/paper/2508.17126/) (2025)
- [Temporal Preference Concepts and their Functions in a Large Language Model](https://mlsanity.com/paper/2606.05194/) (2026)
- [Manifold Bandits: Bayesian Curriculum Learning over the Latent Geometry of Large Language Models](https://mlsanity.com/paper/2606.19750/) (2026)
- [surprisal is Not a Theory](https://mlsanity.com/paper/2607.20208/) (2026)
- [Representation Measurements Under Function-Preserving Reparameterizations](https://mlsanity.com/paper/2608.27020/) (2026)

## Abstract (arXiv)

By virtue of linguistic compositionality, few syntactic rules and a finite lexicon can generate an unbounded number of sentences. That is, language, though seemingly high-dimensional, can be explained using relatively few degrees of freedom. An open question is whether contemporary language models (LMs) reflect the intrinsic simplicity of language that is enabled by compositionality. We take a geometric view of this problem by relating the degree of compositionality in a dataset to the intrinsic dimension (ID) of its representations under an LM, a measure of feature complexity. We find not only that the degree of dataset compositionality is reflected in representations' ID, but that the relationship between compositionality and geometric complexity arises due to learned linguistic features over training. Finally, our analyses reveal a striking contrast between nonlinear and linear dimensionality, showing they respectively encode semantic and superficial aspects of linguistic composition.