The Lattice Representation Hypothesis of Large Language Models
Not on the map. Why?
Abstract (arXiv)
We propose the Lattice Representation Hypothesis of large language models: a symbolic backbone that grounds conceptual hierarchies and logical operations in embedding geometry. Our framework unifies the Linear Representation Hypothesis with Formal Concept Analysis (FCA), showing that linear attribute directions with separating thresholds induce a concept lattice via half-space intersections. This geometry enables symbolic reasoning through geometric meet (intersection) and join (union) operations, and admits a canonical form when attribute directions are linearly independent. Experiments on WordNet sub-hierarchies provide empirical evidence that LLM embeddings encode concept lattices and their logical structure, revealing a principled bridge between continuous geometry and symbolic abstraction. Datasets and code are open available at https://github.com/xiongbo010/lattice-representation-hypothesis.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 22 of its 30 references that are in the map. First posted on arXiv 1 Mar 2026.
Builds on
22 of 30 references.
References · 22
- 2023GPT-4 Technical Reportcited by 18,132
- 2024The Llama 3 Herd of Modelscited by 14,192
- 2019Language Models as Knowledge Bases?cited by 1,922
- 2022Toy Models of Superpositioncited by 928
- 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 679
- 2023Emergent Linear Representations in World Models of Self-Supervised Sequence Modelscited by 313
- 2023Language Models Represent Space and Timecited by 306
- 2024Not All Language Model Features Are One-Dimensionally Linearcited by 201
- 2016Linear Algebraic Structure of Word Senses, with Applications to Polysemycited by 165
- 2024The Geometry of Categorical and Hierarchical Concepts in Large Language Modelscited by 151
- 2024On the Origins of Linear Representations in Large Language Modelscited by 76
- 2023Word Embeddings Are Steers for Language Modelscited by 70
10 more
- 2022Discovering Latent Concepts Learned in BERTcited by 56
- 2022Analyzing Encoded Concepts in Transformer Language Modelscited by 37
- 2020Asking without Telling: Exploring Latent Ontologies in Contextual Representationscited by 34
- 2024Beyond Single Concept Vector: Modeling Concept Subspace in LLMs with Gaussian Distributioncited by 24
- 2025Beyond Linear Steering: Unified Multi-Attribute Control for Language Modelscited by 23
- 2021Inspecting the concept knowledge graph encoded by modern language modelscited by 15
- 2023Do PLMs Know and Understand Ontological Knowledge?cited by 13
- 2023Scaling up Discovery of Latent Concepts in Deep NLP Modelscited by 9
- 2024The Persian Rug: solving toy models of superposition using large-scale symmetriescited by 3
- 2025From Tokens to Lattices: Emergent Lattice Structures in Language Modelscited by 2
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2025The Geometry of Reasoning: Flowing Logics in Representation Space
- 2026Beneath the Surface of Chains-of-Thought: A Mechanistic Interpretation of Reasoning Operations in LLMs
- 2024Reasoning in Large Language Models: A Geometric Perspective
- 2026Unveiling the Reasoning Process of Large Language Models
- 2025Explainable Mapper: Charting LLM Embedding Spaces Using Perturbation-Based Explanation and Verification Agents
- 2025Out-of-Context Reasoning in Large Language Models
- 2026Emergent Manifold Separability during Reasoning in Large Language Models
- 2026Beyond Language: Format-Agnostic Reasoning Subspaces in Large Language Models