# Robust High-dimensional Memory-augmented Neural Networks

arXiv: https://arxiv.org/abs/2010.01939 · HTML: https://mlsanity.com/paper/2010.01939/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Oct 2020.

## Built on by (most cited first, 30 of 31)

- [A Survey on Hyperdimensional Computing aka Vector Symbolic Architectures, Part I: Models and Data Transformations](https://mlsanity.com/paper/2111.06077/) (2021)
- [Constrained Few-shot Class-incremental Learning](https://mlsanity.com/paper/2203.16588/) (2022)
- [A Survey on Hyperdimensional Computing aka Vector Symbolic Architectures, Part II: Applications, Cognitive Models, and Challenges](https://mlsanity.com/paper/2112.15424/) (2021)
- [A Neuro-vector-symbolic Architecture for Solving Raven's Progressive Matrices](https://mlsanity.com/paper/2203.04571/) (2022)
- [Dual Memory Networks: A Versatile Adaptation Approach for Vision-Language Models](https://mlsanity.com/paper/2403.17589/) (2024)
- [A TinyML Platform for On-Device Continual Learning with Quantized Latent Replays](https://mlsanity.com/paper/2110.10486/) (2021)
- [Vector Symbolic Architectures as a Computing Framework for Emerging Hardware](https://mlsanity.com/paper/2106.05268/) (2021)
- [AdaNeg: Adaptive Negative Proxy Guided OOD Detection with Vision-Language Models](https://mlsanity.com/paper/2410.20149/) (2024)
- [Non-Exemplar Online Class-incremental Continual Learning via Dual-prototype Self-augment and Refinement](https://mlsanity.com/paper/2303.10891/) (2023)
- [In-memory Realization of In-situ Few-shot Continual Learning with a Dynamically Evolving Explicit Memory](https://mlsanity.com/paper/2207.06810/) (2022)
- [Hyperseed: Unsupervised Learning with Vector Symbolic Architectures](https://mlsanity.com/paper/2110.08343/) (2021)
- [Learning from Hypervectors: A Survey on Hypervector Encoding](https://mlsanity.com/paper/2308.00685/) (2023)
- [PECAN: A Product-Quantized Content Addressable Memory Network](https://mlsanity.com/paper/2208.13571/) (2022)
- [Spiking Hyperdimensional Network: Neuromorphic Models Integrated with Memory-Inspired Framework](https://mlsanity.com/paper/2110.00214/) (2021)
- [Random Quantum Neural Networks (RQNN) for Noisy Image Recognition](https://mlsanity.com/paper/2203.01764/) (2022)
- [Factorizers for Distributed Sparse Block Codes](https://mlsanity.com/paper/2303.13957/) (2023)
- [Vector-Symbolic Architecture for Event-Based Optical Flow](https://mlsanity.com/paper/2405.08300/) (2024)
- [12 mJ per Class On-Device Online Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2403.07851/) (2024)
- [Generalized Key-Value Memory to Flexibly Adjust Redundancy in Memory-Augmented Networks](https://mlsanity.com/paper/2203.06223/) (2022)
- [The Hyperdimensional Transform for Distributional Modelling, Regression and Classification](https://mlsanity.com/paper/2311.08150/) (2023)
- [H3DFact: Heterogeneous 3D Integrated CIM for Factorization with Holographic Perceptual Representations](https://mlsanity.com/paper/2404.04173/) (2024)
- [MetaWearS: A Shortcut in Wearable Systems Lifecycle with Only a Few Shots](https://mlsanity.com/paper/2408.01988/) (2024)
- [Unsupervised Cognition](https://mlsanity.com/paper/2409.18624/) (2024)
- [Eigen Memory Trees](https://mlsanity.com/paper/2210.14077/) (2022)
- [Pruning random resistive memory for optimizing analogue AI](https://mlsanity.com/paper/2311.07164/) (2023)
- [TLCE: Transfer-Learning Based Classifier Ensembles for Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2312.04225/) (2023)
- [Zero-shot Classification using Hyperdimensional Computing](https://mlsanity.com/paper/2401.16876/) (2024)
- [Learning-to-learn enables rapid learning with phase-change memory-based in-memory computing](https://mlsanity.com/paper/2405.05141/) (2024)
- [Dynamic neural network with memristive CIM and CAM for 2D and 3D vision](https://mlsanity.com/paper/2407.08990/) (2024)
- [Efficient and Reliable Vector Similarity Search Using Asymmetric Encoding with NAND-Flash for Many-Class Few-Shot Learning](https://mlsanity.com/paper/2409.07832/) (2024)

## Abstract (arXiv)

Traditional neural networks require enormous amounts of data to build their complex mappings during a slow training procedure that hinders their abilities for relearning and adapting to new data. Memory-augmented neural networks enhance neural networks with an explicit memory to overcome these issues. Access to this explicit memory, however, occurs via soft read and write operations involving every individual memory entry, resulting in a bottleneck when implemented using the conventional von Neumann computer architecture. To overcome this bottleneck, we propose a robust architecture that employs a computational memory unit as the explicit memory performing analog in-memory computation on high-dimensional (HD) vectors, while closely matching 32-bit software-equivalent accuracy. This is achieved by a content-based attention mechanism that represents unrelated items in the computational memory with uncorrelated HD vectors, whose real-valued components can be readily approximated by binary, or bipolar components. Experimental results demonstrate the efficacy of our approach on few-shot image classification tasks on the Omniglot dataset using more than 256,000 phase-change memory devices. Our approach effectively merges the richness of deep neural network representations with HD computing that paves the way for robust vector-symbolic manipulations applicable in reasoning, fusion, and compression.