# A Geometry-Based Capacity Theory for Finite-Feature Associative Memory

arXiv: https://arxiv.org/abs/2610.09056 · HTML: https://mlsanity.com/paper/2610.09056/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 4 of its 70 references that are in the map.
- First posted on arXiv 6 Oct 2026.

## Builds on (most cited first, 4 of 4)

- [LoLA: Low-Rank Linear Attention With Sparse Caching](https://mlsanity.com/paper/2505.23666/) (2025)
- [Sharp Capacity Thresholds in Linear Associative Memory: From Top-1 Retrieval to Tail-Average Learning](https://mlsanity.com/paper/2605.05189/) (2026)
- [Kernelized Linear Attention: Breaking the Capacity Wall with Symmetric Cones](https://mlsanity.com/paper/2607.17419/) (2026)
- [MLPs are Hebbians: Constructing Efficient Fact-Storing MLPs for Transformers](https://mlsanity.com/paper/2607.10034/) (2026)

## Abstract (arXiv)

We develop a geometry-based capacity theory for exact-key retrieval in compressed finite-feature Hebbian associative memory. For random or approximately isotropic values, retrieval interference separates into finite-feature noise, which decreases with feature dimension, and structural interference, which is determined by squared kernel overlap among stored keys and persists in the infinite-feature limit. This yields a fit-free prediction of retrieval quality, reveals a geometry-dependent capacity ceiling, and predicts the feature budget required for a target retrieval quality. When stored values are correlated, we show that retrieval depends jointly on the key kernel and value Gram matrix, and derive finite-feature approximations that account for this interaction. We validate the theory on synthetic, visual, and medical-image representations. Overall, the framework links representation geometry directly to memory capacity and distinguishes when performance can be improved by increasing the feature budget and when the representation itself must be changed. Across these settings, the predicted retrieval curves closely match empirical behavior and correctly identify changes in the preferred memory design.