# A Walsh Hadamard Derived Linear Vector Symbolic Architecture

arXiv: https://arxiv.org/abs/2410.22669 · HTML: https://mlsanity.com/paper/2410.22669/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 45 references that are in the map.
- First posted on arXiv 30 Oct 2024.

## Builds on (most cited first, 22 of 22)

- [Linear Transformers Are Secretly Fast Weight Programmers](https://mlsanity.com/paper/2102.11174/) (2021)
- [On the Binding Problem in Artificial Neural Networks](https://mlsanity.com/paper/2012.05208/) (2020)
- [AttentionXML: Label Tree-based Attention-Aware Deep Model for High-Performance Extreme Multi-Label Text Classification](https://mlsanity.com/paper/1811.01727/) (2018)
- [A Survey on Hyperdimensional Computing aka Vector Symbolic Architectures, Part I: Models and Data Transformations](https://mlsanity.com/paper/2111.06077/) (2021)
- [A Survey on Hyperdimensional Computing aka Vector Symbolic Architectures, Part II: Applications, Cognitive Models, and Challenges](https://mlsanity.com/paper/2112.15424/) (2021)
- [Efficient softmax approximation for GPUs](https://mlsanity.com/paper/1609.04309/) (2016)
- [A comparison of Vector Symbolic Architectures](https://mlsanity.com/paper/2001.11797/) (2020)
- [InteractE: Improving Convolution-based Knowledge Graph Embeddings by Increasing Feature Interactions](https://mlsanity.com/paper/1911.00219/) (2019)
- [Vector Symbolic Architectures as a Computing Framework for Emerging Hardware](https://mlsanity.com/paper/2106.05268/) (2021)
- [Learning to Reason with Third-Order Tensor Products](https://mlsanity.com/paper/1811.12143/) (2018)
- [Learning with Holographic Reduced Representations](https://mlsanity.com/paper/2109.02157/) (2021)
- [Torchhd: An Open Source Python Library to Support Research on Hyperdimensional Computing and Vector Symbolic Architectures](https://mlsanity.com/paper/2205.09208/) (2022)
- [Learning to Attend via Word-Aspect Associative Fusion for Aspect-based Sentiment Analysis](https://mlsanity.com/paper/1712.05403/) (2017)
- [Recasting Self-Attention with Holographic Reduced Representations](https://mlsanity.com/paper/2305.19534/) (2023)
- [MIMONets: Multiple-Input-Multiple-Output Neural Networks Exploiting Computation in Superposition](https://mlsanity.com/paper/2312.02829/) (2023)
- [CircConv: A Structured Convolution with Low Complexity](https://mlsanity.com/paper/1902.11268/) (2019)
- [Accelerating Extreme Classification via Adaptive Feature Agglomeration](https://mlsanity.com/paper/1905.11769/) (2019)
- [Deploying Convolutional Networks on Untrusted Platforms Using 2D Holographic Reduced Representations](https://mlsanity.com/paper/2206.05893/) (2022)
- [Holographic Global Convolutional Networks for Long-Range Prediction Tasks in Malware Detection](https://mlsanity.com/paper/2403.17978/) (2024)
- [Towards Generalization in Subitizing with Neuro-Symbolic Loss using Holographic Reduced Representations](https://mlsanity.com/paper/2312.15310/) (2023)
- [Lempel-Ziv Networks](https://mlsanity.com/paper/2211.13250/) (2022)
- [WHYPE: A Scale-Out Architecture with Wireless Over-the-Air Majority for Scalable In-memory Hyperdimensional Computing](https://mlsanity.com/paper/2303.08067/) (2023)

## Built on by (most cited first, 4 of 4)

- [Linearithmic Clean-up for Vector-Symbolic Key-Value Memory with Kroneker Rotation Products](https://mlsanity.com/paper/2506.15793/) (2025)
- [Composing Linear Layers from Irreducibles](https://mlsanity.com/paper/2507.11688/) (2025)
- [A Vector Symbolic Approach to Multiple Instance Learning](https://mlsanity.com/paper/2511.16795/) (2025)
- [Recursive Binding on a Budget: Subspace Carving in Order-p Tensor Memories](https://mlsanity.com/paper/2606.11391/) (2026)

## Abstract (arXiv)

Vector Symbolic Architectures (VSAs) are one approach to developing Neuro-symbolic AI, where two vectors in ℝd are `bound' together to produce a new vector in the same space. VSAs support the commutativity and associativity of this binding operation, along with an inverse operation, allowing one to construct symbolic-style manipulations over real-valued vectors. Most VSAs were developed before deep learning and automatic differentiation became popular and instead focused on efficacy in hand-designed systems. In this work, we introduce the Hadamard-derived linear Binding (HLB), which is designed to have favorable computational efficiency, and efficacy in classic VSA tasks, and perform well in differentiable systems. Code is available at https://github.com/FutureComputing4AI/Hadamard-derived-Linear-Binding