# Sinkhorn Based Associative Memory Retrieval Using Spherical Hellinger Kantorovich Dynamics

arXiv: https://arxiv.org/abs/2603.20656 · HTML: https://mlsanity.com/paper/2603.20656/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 33 of its 52 references that are in the map.
- First posted on arXiv 21 Mar 2026.

## Builds on (most cited first, 30 of 33)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Hopfield Networks is All You Need](https://mlsanity.com/paper/2008.02217/) (2020)
- [Dense Associative Memory for Pattern Recognition](https://mlsanity.com/paper/1606.01164/) (2016)
- [Memorizing Transformers](https://mlsanity.com/paper/2203.08913/) (2022)
- [Augmenting Language Models with Long-Term Memory](https://mlsanity.com/paper/2306.07174/) (2023)
- [Birth of a Transformer: A Memory Viewpoint](https://mlsanity.com/paper/2306.00802/) (2023)
- [Optimal Entropy-Transport problems and a new Hellinger-Kantorovich distance between positive measures](https://mlsanity.com/paper/1508.07941/) (2015)
- [Large Associative Memory Problem in Neurobiology and Machine Learning](https://mlsanity.com/paper/2008.06996/) (2020)
- [CLOOB: Modern Hopfield Networks with InfoLOOB Outperform CLIP](https://mlsanity.com/paper/2110.11316/) (2021)
- [Energy Transformer](https://mlsanity.com/paper/2302.07253/) (2023)
- [Relating transformers to models and neural representations of the hippocampal formation](https://mlsanity.com/paper/2112.04035/) (2021)
- [Universal Hopfield Networks: A General Framework for Single-Shot Associative Memory Models](https://mlsanity.com/paper/2202.04557/) (2022)
- [Modern Hopfield Networks and Attention for Immune Repertoire Classification](https://mlsanity.com/paper/2007.13505/) (2020)
- [Sparse Optimization on Measures with Over-parameterized Gradient Descent](https://mlsanity.com/paper/1907.10300/) (2019)
- [Outlier-Efficient Hopfield Layers for Large Transformer-Based Models](https://mlsanity.com/paper/2404.03828/) (2024)
- [Scaling Laws for Associative Memories](https://mlsanity.com/paper/2310.02984/) (2023)
- [CAMELoT: Towards Large Language Models with Training-Free Consolidated Associative Memory](https://mlsanity.com/paper/2402.13449/) (2024)
- [Conv-CoA: Improving Open-domain Question Answering in Large Language Models via Conversational Chain-of-Action](https://mlsanity.com/paper/2405.17822/) (2024)
- [Simplicial Hopfield networks](https://mlsanity.com/paper/2305.05179/) (2023)
- [Sequential Memory with Temporal Predictive Coding](https://mlsanity.com/paper/2305.11982/) (2023)
- [Long Sequence Hopfield Memory](https://mlsanity.com/paper/2306.04532/) (2023)
- [Nonparametric Modern Hopfield Models](https://mlsanity.com/paper/2404.03900/) (2024)
- [Memory Mosaics](https://mlsanity.com/paper/2405.06394/) (2024)
- [Dreaming neural networks: forgetting spurious memories and reinforcing pure ones](https://mlsanity.com/paper/1810.12217/) (2018)
- [End-to-end Differentiable Clustering with Associative Memories](https://mlsanity.com/paper/2306.03209/) (2023)
- [Content Addressable Memory Without Catastrophic Forgetting by Heteroassociation with a Fixed Scaffold](https://mlsanity.com/paper/2202.00159/) (2022)
- [Dense Hebbian neural networks: a replica symmetric picture of supervised learning](https://mlsanity.com/paper/2212.00606/) (2022)
- [Dense Associative Memory Through the Lens of Random Features](https://mlsanity.com/paper/2410.24153/) (2024)
- [HBert + BiasCorp -- Fighting Racism on the Web](https://mlsanity.com/paper/2104.02242/) (2021)
- [Gradient Flows of Potential Energies in the Geometry of Sinkhorn Divergences](https://mlsanity.com/paper/2511.14278/) (2025)

## Abstract (arXiv)

We propose a dense associative memory for empirical measures (weighted point clouds). Stored patterns and queries are finitely supported probability measures, and retrieval is defined by minimizing a Hopfield-style log-sum-exp energy built from the debiased Sinkhorn divergence. We derive retrieval dynamics as a spherical Hellinger Kantorovich (SHK) gradient flow, which updates both support locations and weights. Discretizing the flow yields a deterministic algorithm that uses Sinkhorn potentials to compute barycentric transport steps and a multiplicative simplex reweighting. Under local separation and PL-type conditions we prove basin invariance, geometric convergence to a local minimizer, and a bound showing the minimizer remains close to the corresponding stored pattern. Under a random pattern model, we further show that these Sinkhorn basins are disjoint with high probability, implying exponential capacity in the ambient dimension. Experiments on synthetic Gaussian point-cloud memories demonstrate robust recovery from perturbed queries versus a Euclidean Hopfield-type baseline.