# Self-Attention Based Semantic Decomposition in Vector Symbolic Architectures

arXiv: https://arxiv.org/abs/2403.13218 · HTML: https://mlsanity.com/paper/2403.13218/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 37 references that are in the map.
- First posted on arXiv 20 Mar 2024.

## Builds on (most cited first, 12 of 12)

- [Large Language Models Cannot Self-Correct Reasoning Yet](https://mlsanity.com/paper/2310.01798/) (2023)
- [Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models](https://mlsanity.com/paper/2309.01219/) (2023)
- [Towards Reasoning in Large Language Models: A Survey](https://mlsanity.com/paper/2212.10403/) (2022)
- [HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models](https://mlsanity.com/paper/2305.11747/) (2023)
- [A Survey on Hyperdimensional Computing aka Vector Symbolic Architectures, Part II: Applications, Cognitive Models, and Challenges](https://mlsanity.com/paper/2112.15424/) (2021)
- [A Neuro-vector-symbolic Architecture for Solving Raven's Progressive Matrices](https://mlsanity.com/paper/2203.04571/) (2022)
- [A Theoretical Perspective on Hyperdimensional Computing](https://mlsanity.com/paper/2010.07426/) (2020)
- [Computing on Functions Using Randomized Vector Representations](https://mlsanity.com/paper/2109.03429/) (2021)
- [A theory of sequence indexing and working memory in recurrent neural networks](https://mlsanity.com/paper/1803.00412/) (2018)
- [GraphHD: Efficient graph classification using hyperdimensional computing](https://mlsanity.com/paper/2205.07826/) (2022)
- [In-memory factorization of holographic perceptual representations](https://mlsanity.com/paper/2211.05052/) (2022)
- [Hyperseed: Unsupervised Learning with Vector Symbolic Architectures](https://mlsanity.com/paper/2110.08343/) (2021)

## Built on by (most cited first, 2 of 2)

- [Hyperdimensional Quantum Factorization](https://mlsanity.com/paper/2406.11889/) (2024)
- [Efficient Vector Symbolic Architectures from Histogram Recovery](https://mlsanity.com/paper/2511.01838/) (2025)

## Abstract (arXiv)

Vector Symbolic Architectures (VSAs) have emerged as a novel framework for enabling interpretable machine learning algorithms equipped with the ability to reason and explain their decision processes. The basic idea is to represent discrete information through high dimensional random vectors. Complex data structures can be built up with operations over vectors such as the "binding" operation involving element-wise vector multiplication, which associates data together. The reverse task of decomposing the associated elements is a combinatorially hard task, with an exponentially large search space. The main algorithm for performing this search is the resonator network, inspired by Hopfield network-based memory search operations. In this work, we introduce a new variant of the resonator network, based on self-attention based update rules in the iterative search problem. This update rule, based on the Hopfield network with log-sum-exp energy function and norm-bounded states, is shown to substantially improve the performance and rate of convergence. As a result, our algorithm enables a larger capacity for associative memory, enabling applications in many tasks like perception based pattern recognition, scene decomposition, and object reasoning. We substantiate our algorithm with a thorough evaluation and comparisons to baselines.