# From Data Statistics to Feature Geometry: How Correlations Shape Superposition

arXiv: https://arxiv.org/abs/2603.09972 · HTML: https://mlsanity.com/paper/2603.09972/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Placed in Sparse autoencoder training and evaluation, one of 13 directions in Mechanistic interpretability.
- Builds on 23 of its 45 references that are in the map.
- First posted on arXiv 10 Mar 2026.

## Placed in

- Mechanistic interpretability / [Sparse autoencoder training and evaluation](https://mlsanity.com/mechinterp/sparse-autoencoder-training-and-evaluation/)

## Builds on (most cited first, 23 of 23)

- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Pointer Sentinel Mixture Models](https://mlsanity.com/paper/1609.07843/) (2016)
- [Sparse Autoencoders Find Highly Interpretable Features in Language Models](https://mlsanity.com/paper/2309.08600/) (2023)
- [Toy Models of Superposition](https://mlsanity.com/paper/2209.10652/) (2022)
- [Progress measures for grokking via mechanistic interpretability](https://mlsanity.com/paper/2301.05217/) (2023)
- [The Linear Representation Hypothesis and the Geometry of Large Language Models](https://mlsanity.com/paper/2311.03658/) (2023)
- [Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets](https://mlsanity.com/paper/2201.02177/) (2022)
- [Scaling and evaluating sparse autoencoders](https://mlsanity.com/paper/2406.04093/) (2024)
- [Finding Neurons in a Haystack: Case Studies with Sparse Probing](https://mlsanity.com/paper/2305.01610/) (2023)
- [Language Models Represent Space and Time](https://mlsanity.com/paper/2310.02207/) (2023)
- [Towards Understanding Grokking: An Effective Theory of Representation Learning](https://mlsanity.com/paper/2205.10343/) (2022)
- [Not All Language Model Features Are One-Dimensionally Linear](https://mlsanity.com/paper/2405.14860/) (2024)
- [The Geometry of Categorical and Hierarchical Concepts in Large Language Models](https://mlsanity.com/paper/2406.01506/) (2024)
- [Polysemanticity and Capacity in Neural Networks](https://mlsanity.com/paper/2210.01892/) (2022)
- [Analogies Explained: Towards Understanding Word Embeddings](https://mlsanity.com/paper/1901.09813/) (2019)
- [Projecting Assumptions: The Duality Between Sparse Autoencoders and Concept Geometry](https://mlsanity.com/paper/2503.01822/) (2025)
- [The Geometry of Concepts: Sparse Autoencoder Feature Structure](https://mlsanity.com/paper/2410.19750/) (2024)
- [The Origins of Representation Manifolds in Large Language Models](https://mlsanity.com/paper/2505.18235/) (2025)
- [Shared Global and Local Geometry of Language Model Embeddings](https://mlsanity.com/paper/2503.21073/) (2025)
- [Implicit Geometry of Next-token Prediction: From Language Sparsity Patterns to Model Representations](https://mlsanity.com/paper/2408.15417/) (2024)
- [Representation Shattering in Transformers: A Synthetic Study with Knowledge Editing](https://mlsanity.com/paper/2410.17194/) (2024)
- [Adversarial Examples Are Not Bugs, They Are Superposition](https://mlsanity.com/paper/2508.17456/) (2025)
- [Adversarial Attacks Leverage Interference Between Features in Superposition](https://mlsanity.com/paper/2510.11709/) (2025)

## Built on by (most cited first, 10 of 10)

- [The Scaling Properties of Implicit Deductive Reasoning in Transformers](https://mlsanity.com/paper/2605.04330/) (2026)
- [Feature Superposition in Neural Networks: From Theory to Practice](https://mlsanity.com/paper/2609.06862/) (2026)
- [Adversarial Concept Search: Predicting Compositional Errors From Feature Geometry](https://mlsanity.com/paper/2606.13934/) (2026)
- [Size Doesn't Matter: Cosine-Scored Sparse Autoencoders](https://mlsanity.com/paper/2606.15054/) (2026)
- [GEMS: Geometric Constraints Enable Multi-Semantic Superposition in LLMs](https://mlsanity.com/paper/2606.19946/) (2026)
- [High-probability guarantees for linear accessibility in feature superposition](https://mlsanity.com/paper/2609.09556/) (2026)
- [World Modeling in Transformers](https://mlsanity.com/paper/2609.21748/) (2026)
- [Abstention and Noise Filtering: Two Missing Primitives of Softmax Attention](https://mlsanity.com/paper/2609.22005/) (2026)
- [Interference Beyond Geometry in Concept Extraction](https://mlsanity.com/paper/2609.35351/) (2026)
- [Causal and Interpretable Structures in LLM Compositional Tasks](https://mlsanity.com/paper/2609.35970/) (2026)

## Abstract (arXiv)

A central idea in mechanistic interpretability is that neural networks represent more features than they have dimensions, arranging them in superposition to form an over-complete basis. This framing has been influential, motivating dictionary learning approaches such as sparse autoencoders. However, superposition has mostly been studied in idealized settings where features are sparse and uncorrelated. In these settings, superposition is typically understood as introducing interference that must be minimized geometrically and filtered out by non-linearities such as ReLUs, yielding local structures like regular polytopes. We show that this account is incomplete for realistic data by introducing Bag-of-Words Superposition (BOWS), a controlled setting to encode binary bag-of-words representations of internet text in superposition. Using BOWS, we find that when features are correlated, interference can be constructive rather than just noise to be filtered out. This is achieved by arranging features according to their co-activation patterns, making interference between active features constructive, while still using ReLUs to avoid false positives. We show that this kind of arrangement is more prevalent in models trained with weight decay and naturally gives rise to semantic clusters and cyclical structures which have been observed in real language models yet were not explained by the standard picture of superposition. Code for this paper can be found at https://github.com/LucasPrietoAl/correlations-feature-geometry.