# Generalized Key-Value Memory to Flexibly Adjust Redundancy in Memory-Augmented Networks

arXiv: https://arxiv.org/abs/2203.06223 · HTML: https://mlsanity.com/paper/2203.06223/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 26 references that are in the map.
- First posted on arXiv 11 Mar 2022.

## Builds on (most cited first, 12 of 12)

- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination](https://mlsanity.com/paper/1805.01978/) (2018)
- [Neural Turing Machines](https://mlsanity.com/paper/1410.5401/) (2014)
- [Memory Networks](https://mlsanity.com/paper/1410.3916/) (2014)
- [A Survey on Hyperdimensional Computing aka Vector Symbolic Architectures, Part I: Models and Data Transformations](https://mlsanity.com/paper/2111.06077/) (2021)
- [A Survey on Hyperdimensional Computing aka Vector Symbolic Architectures, Part II: Applications, Cognitive Models, and Challenges](https://mlsanity.com/paper/2112.15424/) (2021)
- [Improving Generalization via Scalable Neighborhood Component Analysis](https://mlsanity.com/paper/1808.04699/) (2018)
- [In-memory hyperdimensional computing](https://mlsanity.com/paper/1906.01548/) (2019)
- [Robust High-dimensional Memory-augmented Neural Networks](https://mlsanity.com/paper/2010.01939/) (2020)
- [A theory of sequence indexing and working memory in recurrent neural networks](https://mlsanity.com/paper/1803.00412/) (2018)
- [Robust computation with rhythmic spike patterns](https://mlsanity.com/paper/1901.07718/) (2019)
- [Neural Distributed Autoassociative Memories: A Survey](https://mlsanity.com/paper/1709.00848/) (2017)

## Built on by (most cited first, 2 of 2)

- [Constrained Few-shot Class-incremental Learning](https://mlsanity.com/paper/2203.16588/) (2022)
- [Contrastive Augmented Graph2Graph Memory Interaction for Few Shot Continual Learning](https://mlsanity.com/paper/2403.04140/) (2024)

## Abstract (arXiv)

Memory-augmented neural networks enhance a neural network with an external key-value memory whose complexity is typically dominated by the number of support vectors in the key memory. We propose a generalized key-value memory that decouples its dimension from the number of support vectors by introducing a free parameter that can arbitrarily add or remove redundancy to the key memory representation. In effect, it provides an additional degree of freedom to flexibly control the trade-off between robustness and the resources required to store and compute the generalized key-value memory. This is particularly useful for realizing the key memory on in-memory computing hardware where it exploits nonideal, but extremely efficient non-volatile memory devices for dense storage and computation. Experimental results show that adapting this parameter on demand effectively mitigates up to 44% nonidealities, at equal accuracy and number of devices, without any need for neural network retraining.