# A Survey on Hyperdimensional Computing aka Vector Symbolic Architectures, Part I: Models and Data Transformations

arXiv: https://arxiv.org/abs/2111.06077 · HTML: https://mlsanity.com/paper/2111.06077/

## Key facts

- Cited by 99 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 200 references that are in the map.
- First posted on arXiv 11 Nov 2021.

## Builds on (most cited first, 25 of 25)

- [Energy and Policy Considerations for Deep Learning in NLP](https://mlsanity.com/paper/1906.02243/) (2019)
- [Holographic Embeddings of Knowledge Graphs](https://mlsanity.com/paper/1510.04935/) (2015)
- [A comparison of Vector Symbolic Architectures](https://mlsanity.com/paper/2001.11797/) (2020)
- [Classification using Hyperdimensional Computing: A Review](https://mlsanity.com/paper/2004.11204/) (2020)
- [Blessing of dimensionality: mathematical foundations of the statistical physics of data](https://mlsanity.com/paper/1801.03421/) (2018)
- [In-memory hyperdimensional computing](https://mlsanity.com/paper/1906.01548/) (2019)
- [A Theoretical Perspective on Hyperdimensional Computing](https://mlsanity.com/paper/2010.07426/) (2020)
- [Computing on Functions Using Randomized Vector Representations](https://mlsanity.com/paper/2109.03429/) (2021)
- [Associative Long Short-Term Memory](https://mlsanity.com/paper/1602.03032/) (2016)
- [Robust High-dimensional Memory-augmented Neural Networks](https://mlsanity.com/paper/2010.01939/) (2020)
- [A theory of sequence indexing and working memory in recurrent neural networks](https://mlsanity.com/paper/1803.00412/) (2018)
- [Variable Binding for Sparse Distributed Representations: Theory and Applications](https://mlsanity.com/paper/2009.06734/) (2020)
- [Learning with Holographic Reduced Representations](https://mlsanity.com/paper/2109.02157/) (2021)
- [Representing Objects, Relations, and Sequences](https://mlsanity.com/paper/1501.07627/) (2015)
- [Robust computation with rhythmic spike patterns](https://mlsanity.com/paper/1901.07718/) (2019)
- [Hardware Optimizations of Dense Binary Hyperdimensional Computing: Rematerialization of Hypervectors, Binarized Bundling, and Combinational Associative Memory](https://mlsanity.com/paper/1807.08583/) (2018)
- [Autoscaling Bloom Filter: Controlling Trade-off Between True and False Positives](https://mlsanity.com/paper/1705.03934/) (2017)
- [Neural Distributed Autoassociative Memories: A Survey](https://mlsanity.com/paper/1709.00848/) (2017)
- [Holographic Graph Neuron: a Bio-Inspired Architecture for Pattern Processing](https://mlsanity.com/paper/1501.03784/) (2015)
- [Encoding Data for HTM Systems](https://mlsanity.com/paper/1602.05925/) (2016)
- [Commentaries on "Learning Sensorimotor Control with Neuromorphic Sensors: Toward Hyperdimensional Active Perception" [Science Robotics Vol. 4 Issue 30 (2019) 1-10](https://mlsanity.com/paper/2003.11458/) (2020)
- [Cellular Automata Can Reduce Memory Requirements of Collective-State Computing](https://mlsanity.com/paper/2010.03585/) (2020)
- [A 5 μ W Standard Cell Memory-based Configurable Hyperdimensional Computing Accelerator for Always-on Smart Sensing](https://mlsanity.com/paper/2102.02758/) (2021)
- [Representing Sets as Summed Semantic Vectors](https://mlsanity.com/paper/1809.08823/) (2018)
- [Analyzing the Capacity of Distributed Vector Representations to Encode Spatial Information](https://mlsanity.com/paper/2010.00055/) (2020)

## Built on by (most cited first, 30 of 99)

- [A Survey on Hyperdimensional Computing aka Vector Symbolic Architectures, Part II: Applications, Cognitive Models, and Challenges](https://mlsanity.com/paper/2112.15424/) (2021)
- [A Neuro-vector-symbolic Architecture for Solving Raven's Progressive Matrices](https://mlsanity.com/paper/2203.04571/) (2022)
- [Modeling rapid language learning by distilling Bayesian priors into artificial neural networks](https://mlsanity.com/paper/2305.14701/) (2023)
- [Deep Learning Methods for Abstract Visual Reasoning: A Survey on Raven's Progressive Matrices](https://mlsanity.com/paper/2201.12382/) (2022)
- [Torchhd: An Open Source Python Library to Support Research on Hyperdimensional Computing and Vector Symbolic Architectures](https://mlsanity.com/paper/2205.09208/) (2022)
- [Neuromorphic Visual Scene Understanding with Resonator Networks](https://mlsanity.com/paper/2208.12880/) (2022)
- [Imperative Learning: A Self-supervised Neuro-Symbolic Learning Framework for Robot Autonomy](https://mlsanity.com/paper/2406.16087/) (2024)
- [HDC-MiniROCKET: Explicit Time Encoding in Time Series Classification with Hyperdimensional Computing](https://mlsanity.com/paper/2202.08055/) (2022)
- [MIMONets: Multiple-Input-Multiple-Output Neural Networks Exploiting Computation in Superposition](https://mlsanity.com/paper/2312.02829/) (2023)
- [Unpaired Image Translation via Vector Symbolic Architectures](https://mlsanity.com/paper/2209.02686/) (2022)
- [In-memory factorization of holographic perceptual representations](https://mlsanity.com/paper/2211.05052/) (2022)
- [Probabilistic Abduction for Visual Abstract Reasoning via Learning Rules in Vector-symbolic Architectures](https://mlsanity.com/paper/2401.16024/) (2024)
- [Towards Learning Abductive Reasoning using VSA Distributed Representations](https://mlsanity.com/paper/2406.19121/) (2024)
- [Learning from Hypervectors: A Survey on Hypervector Encoding](https://mlsanity.com/paper/2308.00685/) (2023)
- [Computing with Residue Numbers in High-Dimensional Representation](https://mlsanity.com/paper/2311.04872/) (2023)
- [Collapse-Proof Non-Contrastive Self-Supervised Learning](https://mlsanity.com/paper/2410.04959/) (2024)
- [Towards Learning to Reason: Comparing LLMs with Neuro-Symbolic on Arithmetic Relations in Abstract Reasoning](https://mlsanity.com/paper/2412.05586/) (2024)
- [Generalized Holographic Reduced Representations](https://mlsanity.com/paper/2405.09689/) (2024)
- [Differentiable Tree Operations Promote Compositional Generalization](https://mlsanity.com/paper/2306.00751/) (2023)
- [An Encoding Framework for Binarized Images using HyperDimensional Computing](https://mlsanity.com/paper/2312.00454/) (2023)
- [Hyperdimensional Representation Learning for Node Classification and Link Prediction](https://mlsanity.com/paper/2402.17073/) (2024)
- [Compositional Generalization Across Distributional Shifts with Sparse Tree Operations](https://mlsanity.com/paper/2412.14076/) (2024)
- [Shift-Equivariant Similarity-Preserving Hypervector Representations of Sequences](https://mlsanity.com/paper/2112.15475/) (2021)
- [Factorizers for Distributed Sparse Block Codes](https://mlsanity.com/paper/2303.13957/) (2023)
- [Laplace-HDC: Understanding the geometry of binary hyperdimensional computing](https://mlsanity.com/paper/2404.10759/) (2024)
- [Vector-Symbolic Architecture for Event-Based Optical Flow](https://mlsanity.com/paper/2405.08300/) (2024)
- [A Walsh Hadamard Derived Linear Vector Symbolic Architecture](https://mlsanity.com/paper/2410.22669/) (2024)
- [Dynamic Epistemic Friction in Dialogue](https://mlsanity.com/paper/2506.10934/) (2025)
- [Streaming Encoding Algorithms for Scalable Hyperdimensional Computing](https://mlsanity.com/paper/2209.09868/) (2022)
- [Capacity Analysis of Vector Symbolic Architectures](https://mlsanity.com/paper/2301.10352/) (2023)

## Abstract (arXiv)

This two-part comprehensive survey is devoted to a computing framework most commonly known under the names Hyperdimensional Computing and Vector Symbolic Architectures (HDC/VSA). Both names refer to a family of computational models that use high-dimensional distributed representations and rely on the algebraic properties of their key operations to incorporate the advantages of structured symbolic representations and vector distributed representations. Notable models in the HDC/VSA family are Tensor Product Representations, Holographic Reduced Representations, Multiply-Add-Permute, Binary Spatter Codes, and Sparse Binary Distributed Representations but there are other models too. HDC/VSA is a highly interdisciplinary field with connections to computer science, electrical engineering, artificial intelligence, mathematics, and cognitive science. This fact makes it challenging to create a thorough overview of the field. However, due to a surge of new researchers joining the field in recent years, the necessity for a comprehensive survey of the field has become extremely important. Therefore, amongst other aspects of the field, this Part I surveys important aspects such as: known computational models of HDC/VSA and transformations of various input data types to high-dimensional distributed representations. Part II of this survey is devoted to applications, cognitive computing and architectures, as well as directions for future work. The survey is written to be useful for both newcomers and practitioners.