# Product Kanerva Machines: Factorized Bayesian Memory

arXiv: https://arxiv.org/abs/2002.02385 · HTML: https://mlsanity.com/paper/2002.02385/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Feb 2020.

## Built on by (most cited first, 5 of 5)

- [Attention Approximates Sparse Distributed Memory](https://mlsanity.com/paper/2111.05498/) (2021)
- [Kanerva++: extending The Kanerva Machine with differentiable, locally block allocated latent memory](https://mlsanity.com/paper/2103.03905/) (2021)
- [BAM: Bayes with Adaptive Memory](https://mlsanity.com/paper/2202.02405/) (2022)
- [Meta-Learning with Variational Bayes](https://mlsanity.com/paper/2103.02265/) (2021)
- [DynaLay: An Introspective Approach to Dynamic Layer Selection for Deep Networks](https://mlsanity.com/paper/2312.12781/) (2023)

## Abstract (arXiv)

An ideal cognitively-inspired memory system would compress and organize incoming items. The Kanerva Machine (Wu et al, 2018) is a Bayesian model that naturally implements online memory compression. However, the organization of the Kanerva Machine is limited by its use of a single Gaussian random matrix for storage. Here we introduce the Product Kanerva Machine, which dynamically combines many smaller Kanerva Machines. Its hierarchical structure provides a principled way to abstract invariant features and gives scaling and capacity advantages over single Kanerva Machines. We show that it can exhibit unsupervised clustering, find sparse and combinatorial allocation patterns, and discover spatial tunings that approximately factorize simple images by object.