# Vector Symbolic Architectures as a Computing Framework for Emerging Hardware

arXiv: https://arxiv.org/abs/2106.05268 · HTML: https://mlsanity.com/paper/2106.05268/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 213 references that are in the map.
- First posted on arXiv 9 Jun 2021.

## Builds on (most cited first, 30 of 39)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Energy and Policy Considerations for Deep Learning in NLP](https://mlsanity.com/paper/1906.02243/) (2019)
- [On the Binding Problem in Artificial Neural Networks](https://mlsanity.com/paper/2012.05208/) (2020)
- [Holographic Embeddings of Knowledge Graphs](https://mlsanity.com/paper/1510.04935/) (2015)
- [All-Optical Machine Learning Using Diffractive Deep Neural Networks](https://mlsanity.com/paper/1804.08711/) (2018)
- [Demystifying Parallel and Distributed Deep Learning: An In-Depth Concurrency Analysis](https://mlsanity.com/paper/1802.09941/) (2018)
- [On the Turing Completeness of Modern Neural Network Architectures](https://mlsanity.com/paper/1901.03429/) (2019)
- [Superposition of many models into one](https://mlsanity.com/paper/1902.05522/) (2019)
- [A comparison of Vector Symbolic Architectures](https://mlsanity.com/paper/2001.11797/) (2020)
- [Classification using Hyperdimensional Computing: A Review](https://mlsanity.com/paper/2004.11204/) (2020)
- [Blessing of dimensionality: mathematical foundations of the statistical physics of data](https://mlsanity.com/paper/1801.03421/) (2018)
- [In-memory hyperdimensional computing](https://mlsanity.com/paper/1906.01548/) (2019)
- [A Theoretical Perspective on Hyperdimensional Computing](https://mlsanity.com/paper/2010.07426/) (2020)
- [Hyperdimensional computing as a framework for systematic aggregation of image descriptors](https://mlsanity.com/paper/2101.07720/) (2021)
- [Robust High-dimensional Memory-augmented Neural Networks](https://mlsanity.com/paper/2010.01939/) (2020)
- [A theory of sequence indexing and working memory in recurrent neural networks](https://mlsanity.com/paper/1803.00412/) (2018)
- [Variable Binding for Sparse Distributed Representations: Theory and Applications](https://mlsanity.com/paper/2009.06734/) (2020)
- [Representing Objects, Relations, and Sequences](https://mlsanity.com/paper/1501.07627/) (2015)
- [Robust computation with rhythmic spike patterns](https://mlsanity.com/paper/1901.07718/) (2019)
- [Rapid online learning and robust recall in a neuromorphic olfactory circuit](https://mlsanity.com/paper/1906.07067/) (2019)
- [Density Encoding Enables Resource-Efficient Randomly Connected Neural Networks](https://mlsanity.com/paper/1909.09153/) (2019)
- [Hardware Optimizations of Dense Binary Hyperdimensional Computing: Rematerialization of Hypervectors, Binarized Bundling, and Combinational Associative Memory](https://mlsanity.com/paper/1807.08583/) (2018)
- [End to End Binarized Neural Networks for Text Classification](https://mlsanity.com/paper/2010.05223/) (2020)
- [Autoscaling Bloom Filter: Controlling Trade-off Between True and False Positives](https://mlsanity.com/paper/1705.03934/) (2017)
- [Neural Distributed Autoassociative Memories: A Survey](https://mlsanity.com/paper/1709.00848/) (2017)
- [Neuromorphic Nearest-Neighbor Search Using Intel's Pohoiki Springs](https://mlsanity.com/paper/2004.12691/) (2020)
- [Integer Echo State Networks: Efficient Reservoir Computing for Digital Hardware](https://mlsanity.com/paper/1706.00280/) (2017)
- [PULP-HD: Accelerating Brain-Inspired High-Dimensional Computing on a Parallel Ultra-Low Power Platform](https://mlsanity.com/paper/1804.09123/) (2018)

## Built on by (most cited first, 30 of 35)

- [A Survey on Hyperdimensional Computing aka Vector Symbolic Architectures, Part II: Applications, Cognitive Models, and Challenges](https://mlsanity.com/paper/2112.15424/) (2021)
- [Torchhd: An Open Source Python Library to Support Research on Hyperdimensional Computing and Vector Symbolic Architectures](https://mlsanity.com/paper/2205.09208/) (2022)
- [MIMONets: Multiple-Input-Multiple-Output Neural Networks Exploiting Computation in Superposition](https://mlsanity.com/paper/2312.02829/) (2023)
- [Hyperseed: Unsupervised Learning with Vector Symbolic Architectures](https://mlsanity.com/paper/2110.08343/) (2021)
- [Learning from Hypervectors: A Survey on Hypervector Encoding](https://mlsanity.com/paper/2308.00685/) (2023)
- [Computing with Residue Numbers in High-Dimensional Representation](https://mlsanity.com/paper/2311.04872/) (2023)
- [Towards Efficient Neuro-Symbolic AI: From Workload Characterization to Hardware Architecture](https://mlsanity.com/paper/2409.13153/) (2024)
- [Generalized Holographic Reduced Representations](https://mlsanity.com/paper/2405.09689/) (2024)
- [Shift-Equivariant Similarity-Preserving Hypervector Representations of Sequences](https://mlsanity.com/paper/2112.15475/) (2021)
- [A Walsh Hadamard Derived Linear Vector Symbolic Architecture](https://mlsanity.com/paper/2410.22669/) (2024)
- [The Discovery Engine: A Framework for AI-Driven Synthesis and Navigation of Scientific Knowledge Landscapes](https://mlsanity.com/paper/2505.17500/) (2025)
- [Compositional Factorization of Visual Scenes with Convolutional Sparse Coding and Resonator Networks](https://mlsanity.com/paper/2404.19126/) (2024)
- [Distributed Representations Enable Robust Multi-Timescale Symbolic Computation in Neuromorphic Hardware](https://mlsanity.com/paper/2405.01305/) (2024)
- [GPT-2 Through the Lens of Vector Symbolic Architectures](https://mlsanity.com/paper/2412.07947/) (2024)
- [Hyperdimensional computing: a fast, robust and interpretable paradigm for biological data](https://mlsanity.com/paper/2402.17572/) (2024)
- [H3DFact: Heterogeneous 3D Integrated CIM for Factorization with Holographic Perceptual Representations](https://mlsanity.com/paper/2404.04173/) (2024)
- [Symbolic Disentangled Representations for Images](https://mlsanity.com/paper/2412.19847/) (2024)
- [HyperCam: Low-Power Onboard Computer Vision for IoT Cameras](https://mlsanity.com/paper/2501.10547/) (2025)
- [HD-CB: The First Exploration of Hyperdimensional Computing for Contextual Bandits Problems](https://mlsanity.com/paper/2501.16863/) (2025)
- [Linearithmic Clean-up for Vector-Symbolic Key-Value Memory with Kroneker Rotation Products](https://mlsanity.com/paper/2506.15793/) (2025)
- [Scheduled Knowledge Acquisition on Lightweight Vector Symbolic Architectures for Brain-Computer Interfaces](https://mlsanity.com/paper/2403.13844/) (2024)
- [HYPERDOA: Robust and Efficient DoA Estimation using Hyperdimensional Computing](https://mlsanity.com/paper/2510.10718/) (2025)
- [Vector Symbolic Algebras for the Abstraction and Reasoning Corpus](https://mlsanity.com/paper/2511.08747/) (2025)
- [Analogical Reasoning Within a Conceptual Hyperspace](https://mlsanity.com/paper/2411.08684/) (2024)
- [Improved Cleanup and Decoding of Fractional Power Encodings](https://mlsanity.com/paper/2412.00488/) (2024)
- [A Grid Cell-Inspired Structured Vector Algebra for Cognitive Maps](https://mlsanity.com/paper/2503.08608/) (2025)
- [Autonomous Learning with High-Dimensional Computing Architecture Similar to von Neumann's](https://mlsanity.com/paper/2503.23608/) (2025)
- [A Vector Symbolic Approach to Multiple Instance Learning](https://mlsanity.com/paper/2511.16795/) (2025)
- [ConformalHDC: Uncertainty-Aware Hyperdimensional Computing with Application to Neural Decoding](https://mlsanity.com/paper/2602.21446/) (2026)
- [Contextual Bandits for Resource-Constrained Devices using Probabilistic Learning](https://mlsanity.com/paper/2605.13346/) (2026)

## Abstract (arXiv)

This article reviews recent progress in the development of the computing framework vector symbolic architectures (VSA) (also known as hyperdimensional computing). This framework is well suited for implementation in stochastic, emerging hardware, and it naturally expresses the types of cognitive operations required for artificial intelligence (AI). We demonstrate in this article that the field-like algebraic structure of VSA offers simple but powerful operations on high-dimensional vectors that can support all data structures and manipulations relevant to modern computing. In addition, we illustrate the distinguishing feature of VSA, "computing in superposition," which sets it apart from conventional computing. It also opens the door to efficient solutions to the difficult combinatorial search problems inherent in AI applications. We sketch ways of demonstrating that VSA are computationally universal. We see them acting as a framework for computing with distributed representations that can play a role of an abstraction layer for emerging computing hardware. This article serves as a reference for computer architects by illustrating the philosophy behind VSA, techniques of distributed computing with them, and their relevance to emerging computing hardware, such as neuromorphic computing.