# On the Role of Noise in Factorizers for Disentangling Distributed Representations

arXiv: https://arxiv.org/abs/2412.00354 · HTML: https://mlsanity.com/paper/2412.00354/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 30 references that are in the map.
- First posted on arXiv 30 Nov 2024.

## Builds on (most cited first, 9 of 9)

- [A Survey on Hyperdimensional Computing aka Vector Symbolic Architectures, Part I: Models and Data Transformations](https://mlsanity.com/paper/2111.06077/) (2021)
- [A Survey on Hyperdimensional Computing aka Vector Symbolic Architectures, Part II: Applications, Cognitive Models, and Challenges](https://mlsanity.com/paper/2112.15424/) (2021)
- [A Neuro-vector-symbolic Architecture for Solving Raven's Progressive Matrices](https://mlsanity.com/paper/2203.04571/) (2022)
- [In-memory hyperdimensional computing](https://mlsanity.com/paper/1906.01548/) (2019)
- [Neuromorphic Visual Scene Understanding with Resonator Networks](https://mlsanity.com/paper/2208.12880/) (2022)
- [In-memory factorization of holographic perceptual representations](https://mlsanity.com/paper/2211.05052/) (2022)
- [Visual Odometry with Neuromorphic Resonator Networks](https://mlsanity.com/paper/2209.02000/) (2022)
- [Integer Factorization with Compositional Distributed Representations](https://mlsanity.com/paper/2203.00920/) (2022)
- [Factorizers for Distributed Sparse Block Codes](https://mlsanity.com/paper/2303.13957/) (2023)

## Built on by (most cited first, 1 of 1)

- [SRMU: Relevance-Gated Updates for Streaming Hyperdimensional Memories](https://mlsanity.com/paper/2604.15121/) (2026)

## Abstract (arXiv)

To efficiently factorize high-dimensional distributed representations to the constituent atomic vectors, one can exploit the compute-in-superposition capabilities of vector-symbolic architectures (VSA). Such factorizers however suffer from the phenomenon of limit cycles. Applying noise during the iterative decoding is one mechanism to address this issue. In this paper, we explore ways to further relax the noise requirement by applying noise only at the time of VSA's reconstruction codebook initialization. While the need for noise during iterations proves analog in-memory computing systems to be a natural choice as an implementation media, the adequacy of initialization noise allows digital hardware to remain equally indispensable. This broadens the implementation possibilities of factorizers. Our study finds that while the best performance shifts from initialization noise to iterative noise as the number of factors increases from 2 to 4, both extend the operational capacity by at least 50 times compared to the baseline factorizer resonator networks. Our code is available at: https://github.com/IBM/in-memory-factorizer