Sinkhorn Based Associative Memory Retrieval Using Spherical Hellinger Kantorovich Dynamics
Not on the map. Why?
Abstract (arXiv)
We propose a dense associative memory for empirical measures (weighted point clouds). Stored patterns and queries are finitely supported probability measures, and retrieval is defined by minimizing a Hopfield-style log-sum-exp energy built from the debiased Sinkhorn divergence. We derive retrieval dynamics as a spherical Hellinger Kantorovich (SHK) gradient flow, which updates both support locations and weights. Discretizing the flow yields a deterministic algorithm that uses Sinkhorn potentials to compute barycentric transport steps and a multiplicative simplex reweighting. Under local separation and PL-type conditions we prove basin invariance, geometric convergence to a local minimizer, and a bound showing the minimizer remains close to the corresponding stored pattern. Under a random pattern model, we further show that these Sinkhorn basins are disjoint with high probability, implying exponential capacity in the ambient dimension. Experiments on synthetic Gaussian point-cloud memories demonstrate robust recovery from perturbed queries versus a Euclidean Hopfield-type baseline.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 33 of its 52 references that are in the map. First posted on arXiv 21 Mar 2026.
Builds on
33 of 52 references.
References · 33
- 2017Attention Is All You Needcited by 59,932
- 2020Hopfield Networks is All You Needcited by 461
- 2016Dense Associative Memory for Pattern Recognitioncited by 230
- 2022Memorizing Transformerscited by 219
- 2023Augmenting Language Models with Long-Term Memorycited by 186
- 2023Birth of a Transformer: A Memory Viewpointcited by 154
- 2015Optimal Entropy-Transport problems and a new Hellinger-Kantorovich distance between positive measurescited by 138
- 2020Large Associative Memory Problem in Neurobiology and Machine Learningcited by 110
- 2021CLOOB: Modern Hopfield Networks with InfoLOOB Outperform CLIPcited by 88
- 2023Energy Transformercited by 69
- 2021Relating transformers to models and neural representations of the hippocampal formationcited by 62
- 2020Modern Hopfield Networks and Attention for Immune Repertoire Classificationcited by 52
21 more
- 2022Universal Hopfield Networks: A General Framework for Single-Shot Associative Memory Modelscited by 52
- 2019Sparse Optimization on Measures with Over-parameterized Gradient Descentcited by 50
- 2024Outlier-Efficient Hopfield Layers for Large Transformer-Based Modelscited by 38
- 2023Scaling Laws for Associative Memoriescited by 30
- 2024CAMELoT: Towards Large Language Models with Training-Free Consolidated Associative Memorycited by 24
- 2024Conv-CoA: Improving Open-domain Question Answering in Large Language Models via Conversational Chain-of-Actioncited by 19
- 2023Simplicial Hopfield networkscited by 16
- 2024Nonparametric Modern Hopfield Modelscited by 16
- 2024Memory Mosaicscited by 16
- 2018Dreaming neural networks: forgetting spurious memories and reinforcing pure onescited by 15
- 2023End-to-end Differentiable Clustering with Associative Memoriescited by 15
- 2023Long Sequence Hopfield Memorycited by 15
- 2023Sequential Memory with Temporal Predictive Codingcited by 14
- 2022Content Addressable Memory Without Catastrophic Forgetting by Heteroassociation with a Fixed Scaffoldcited by 11
- 2022Dense Hebbian neural networks: a replica symmetric picture of supervised learningcited by 10
- 2024Dense Associative Memory Through the Lens of Random Featurescited by 8
- 2021HBert + BiasCorp -- Fighting Racism on the Webcited by 4
- 2023iMixer: hierarchical Hopfield network implies an invertible, implicit and iterative MLP-Mixercited by 3
- 2025Gradient Flows of Potential Energies in the Geometry of Sinkhorn Divergencescited by 3
- 2025Dense associative memory for Gaussian distributionscited by 2
- 2026Deep Clustering with Associative Memoriescited by 2
Built on by
Not cited yet.