Representation Learning on a Random Lattice
Not on the map. Why?
Abstract (arXiv)
Decomposing a deep neural network's learned representations into interpretable features could greatly enhance its safety and reliability. To better understand features, we adopt a geometric perspective, viewing them as a learned coordinate system for mapping an embedded data distribution. We motivate a model of a generic data distribution as a random lattice and analyze its properties using percolation theory. Learned features are categorized into context, component, and surface features. The model is qualitatively consistent with recent findings in mechanistic interpretability and suggests directions for future research.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 23 of its 46 references that are in the map. First posted on arXiv 28 Apr 2025.
Builds on
23 of 46 references.
References · 23
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
- 2017Neural Discrete Representation Learningcited by 4,650
- 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
- 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,359
- 2019Adversarial Examples Are Not Bugs, They Are Featurescited by 950
- 2022Toy Models of Superpositioncited by 928
- 2023Progress measures for grokking via mechanistic interpretabilitycited by 822
- 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 679
- 2024Scaling and evaluating sparse autoencoderscited by 530
- 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 415
- 2024Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2cited by 381
- 2024The Platonic Representation Hypothesiscited by 362
11 more
- 2023Finding Neurons in a Haystack: Case Studies with Sparse Probingcited by 328
- 2022Towards Understanding Grokking: An Effective Theory of Representation Learningcited by 229
- 2023The Clock and the Pizza: Two Stories in Mechanistic Explanation of Neural Networkscited by 153
- 2024A is for Absorption: Studying Feature Splitting and Absorption in Sparse Autoencoderscited by 148
- 2023The Quantization Model of Neural Scalingcited by 137
- 2025Sparse Autoencoders Do Not Find Canonical Units of Analysiscited by 80
- 2025Projecting Assumptions: The Duality Between Sparse Autoencoders and Concept Geometrycited by 58
- 2024The Geometry of Concepts: Sparse Autoencoder Feature Structurecited by 52
- 2024Decomposing The Dark Matter of Sparse Autoencoderscited by 42
- 2019Embedded Agencycited by 16
- 2024Neural Scaling Laws Rooted in the Data Distributioncited by 14
Built on by
Citing papers per year: 2026: 2.
2 papers.