# Single-Layer MeMo as a Randomized Hamming-Kernel Classifier

arXiv: https://arxiv.org/abs/2609.34562 · HTML: https://mlsanity.com/paper/2609.34562/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 17 references that are in the map.
- First posted on arXiv 28 Sep 2026.

## Builds on (most cited first, 7 of 7)

- [Pointer Sentinel Mixture Models](https://mlsanity.com/paper/1609.07843/) (2016)
- [Titans: Learning to Memorize at Test Time](https://mlsanity.com/paper/2501.00663/) (2024)
- [Memory Layers at Scale](https://mlsanity.com/paper/2412.09764/) (2024)
- [Scaling Laws for Associative Memories](https://mlsanity.com/paper/2310.02984/) (2023)
- [Efficient Approximation Algorithms for String Kernel Based Sequence Classification](https://mlsanity.com/paper/1712.04264/) (2017)
- [Biological Sequence Kernels with Guaranteed Flexibility](https://mlsanity.com/paper/2304.03775/) (2023)
- [MeMo: Towards Language Models with Associative Memory Mechanisms](https://mlsanity.com/paper/2502.12851/) (2025)

## Abstract (arXiv)

MeMo (Zanzotto et al., 2025) is a recent language-model architecture that stores associations between token contexts and next tokens in a correlation matrix memory. In this work, we study its single-layer form and show that its ideal retrieval rule is a multiclass classifier based on the positional Hamming kernel. The MeMo architecture represents both the sequence features and the output labels with Gaussian random codes. Its score is therefore a doubly randomized sketch of the ideal classifier. Under independent input and output codebooks, we bound the errors introduced by context sketching and output decoding, characterize their dependence on model and data parameters, and give a margin-based guarantee for recovering the ideal prediction. Controlled simulations support the trends predicted by the analysis. On a restricted WikiText-2 next-token task, we compare single-layer MeMo with classical baselines and show that it can offer a useful trade-off among predictive accuracy, memory, and throughput, particularly on a GPU, where its matrix operations can be parallelized.