# On Design Choices in Similarity-Preserving Sparse Randomized Embeddings

arXiv: https://arxiv.org/abs/2501.14741 · HTML: https://mlsanity.com/paper/2501.14741/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 59 references that are in the map.
- First posted on arXiv 12 Dec 2024.

## Builds on (most cited first, 22 of 22)

- [A Survey on Hyperdimensional Computing aka Vector Symbolic Architectures, Part I: Models and Data Transformations](https://mlsanity.com/paper/2111.06077/) (2021)
- [A Survey on Hyperdimensional Computing aka Vector Symbolic Architectures, Part II: Applications, Cognitive Models, and Challenges](https://mlsanity.com/paper/2112.15424/) (2021)
- [A comparison of Vector Symbolic Architectures](https://mlsanity.com/paper/2001.11797/) (2020)
- [Classification using Hyperdimensional Computing: A Review](https://mlsanity.com/paper/2004.11204/) (2020)
- [A Theoretical Perspective on Hyperdimensional Computing](https://mlsanity.com/paper/2010.07426/) (2020)
- [Computing on Functions Using Randomized Vector Representations](https://mlsanity.com/paper/2109.03429/) (2021)
- [Hyperdimensional computing as a framework for systematic aggregation of image descriptors](https://mlsanity.com/paper/2101.07720/) (2021)
- [A theory of sequence indexing and working memory in recurrent neural networks](https://mlsanity.com/paper/1803.00412/) (2018)
- [Attention Approximates Sparse Distributed Memory](https://mlsanity.com/paper/2111.05498/) (2021)
- [Variable Binding for Sparse Distributed Representations: Theory and Applications](https://mlsanity.com/paper/2009.06734/) (2020)
- [Discrete Key-Value Bottleneck](https://mlsanity.com/paper/2207.11240/) (2022)
- [Can a Fruit Fly Learn Word Embeddings?](https://mlsanity.com/paper/2101.06887/) (2021)
- [Neural Distributed Autoassociative Memories: A Survey](https://mlsanity.com/paper/1709.00848/) (2017)
- [Bio-Inspired Hashing for Unsupervised Similarity Search](https://mlsanity.com/paper/2001.04907/) (2020)
- [Sparse Distributed Memory is a Continual Learner](https://mlsanity.com/paper/2303.11934/) (2023)
- [A survey of dimensionality reduction techniques based on random projection](https://mlsanity.com/paper/1706.04371/) (2017)
- [Expressivity of expand-and-sparsify representations](https://mlsanity.com/paper/2006.03741/) (2020)
- [Laplace-HDC: Understanding the geometry of binary hyperdimensional computing](https://mlsanity.com/paper/2404.10759/) (2024)
- [Streaming Encoding Algorithms for Scalable Hyperdimensional Computing](https://mlsanity.com/paper/2209.09868/) (2022)
- [Capacity Analysis of Vector Symbolic Architectures](https://mlsanity.com/paper/2301.10352/) (2023)
- [Efficient Decoding of Compositional Structure in Holistic Representations](https://mlsanity.com/paper/2305.16873/) (2023)
- [Binary autoencoder with random binary weights](https://mlsanity.com/paper/2004.14717/) (2020)

## Built on by (most cited first, 1 of 1)

- [Large-Margin Hyperdimensional Computing: A Learning-Theoretical Perspective](https://mlsanity.com/paper/2603.03830/) (2026)

## Abstract (arXiv)

Expand & Sparsify is a principle that is observed in anatomically similar neural circuits found in the mushroom body (insects) and the cerebellum (mammals). Sensory data are projected randomly to much higher-dimensionality (expand part) where only few the most strongly excited neurons are activated (sparsify part). This principle has been leveraged to design a FlyHash algorithm that forms similarity-preserving sparse embeddings, which have been found useful for such tasks as novelty detection, pattern recognition, and similarity search. Despite its simplicity, FlyHash has a number of design choices to be set such as preprocessing of the input data, choice of sparsifying activation function, and formation of the random projection matrix. In this paper, we explore the effect of these choices on the performance of similarity search with FlyHash embeddings. We find that the right combination of design choices can lead to drastic difference in the search performance.