# Hyperdimensional Computing for Efficient Distributed Classification with Randomized Neural Networks

arXiv: https://arxiv.org/abs/2106.00881 · HTML: https://mlsanity.com/paper/2106.00881/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 70 references that are in the map.
- First posted on arXiv 2 Jun 2021.

## Builds on (most cited first, 17 of 17)

- [Methods for Interpreting and Understanding Deep Neural Networks](https://mlsanity.com/paper/1706.07979/) (2017)
- [A Survey on Distributed Machine Learning](https://mlsanity.com/paper/1912.09789/) (2019)
- [Superposition of many models into one](https://mlsanity.com/paper/1902.05522/) (2019)
- [Communication-Efficient and Distributed Learning Over Wireless Networks: Principles and Applications](https://mlsanity.com/paper/2008.02608/) (2020)
- [Classification using Hyperdimensional Computing: A Review](https://mlsanity.com/paper/2004.11204/) (2020)
- [Distributed learning with regularized least squares](https://mlsanity.com/paper/1608.03339/) (2016)
- [Deep Randomized Neural Networks](https://mlsanity.com/paper/2002.12287/) (2020)
- [A theory of sequence indexing and working memory in recurrent neural networks](https://mlsanity.com/paper/1803.00412/) (2018)
- [Variable Binding for Sparse Distributed Representations: Theory and Applications](https://mlsanity.com/paper/2009.06734/) (2020)
- [Representing Objects, Relations, and Sequences](https://mlsanity.com/paper/1501.07627/) (2015)
- [Density Encoding Enables Resource-Efficient Randomly Connected Neural Networks](https://mlsanity.com/paper/1909.09153/) (2019)
- [End to End Binarized Neural Networks for Text Classification](https://mlsanity.com/paper/2010.05223/) (2020)
- [Autoscaling Bloom Filter: Controlling Trade-off Between True and False Positives](https://mlsanity.com/paper/1705.03934/) (2017)
- [A review of learning vector quantization classifiers](https://mlsanity.com/paper/1509.07093/) (2015)
- [Integer Echo State Networks: Efficient Reservoir Computing for Digital Hardware](https://mlsanity.com/paper/1706.00280/) (2017)
- [HyperEmbed: Tradeoffs Between Resources and Performance in NLP Tasks with Hyperdimensional Computing enabled Embedding of n-gram Statistics](https://mlsanity.com/paper/2003.01821/) (2020)
- [A Framework for Parallel and Distributed Training of Neural Networks](https://mlsanity.com/paper/1610.07448/) (2016)

## Built on by (most cited first, 7 of 7)

- [A Survey on Hyperdimensional Computing aka Vector Symbolic Architectures, Part II: Applications, Cognitive Models, and Challenges](https://mlsanity.com/paper/2112.15424/) (2021)
- [Vector Symbolic Architectures as a Computing Framework for Emerging Hardware](https://mlsanity.com/paper/2106.05268/) (2021)
- [Generalized Learning Vector Quantization for Classification in Randomized Neural Networks and Hyperdimensional Computing](https://mlsanity.com/paper/2106.09821/) (2021)
- [Hyperseed: Unsupervised Learning with Vector Symbolic Architectures](https://mlsanity.com/paper/2110.08343/) (2021)
- [Learning from Hypervectors: A Survey on Hypervector Encoding](https://mlsanity.com/paper/2308.00685/) (2023)
- [On Effects of Compression with Hyperdimensional Computing in Distributed Randomized Neural Networks](https://mlsanity.com/paper/2106.09831/) (2021)
- [Computing with Hypervectors for Efficient Speaker Identification](https://mlsanity.com/paper/2208.13285/) (2022)

## Abstract (arXiv)

In the supervised learning domain, considering the recent prevalence of algorithms with high computational cost, the attention is steering towards simpler, lighter, and less computationally extensive training and inference approaches. In particular, randomized algorithms are currently having a resurgence, given their generalized elementary approach. By using randomized neural networks, we study distributed classification, which can be employed in situations were data cannot be stored at a central location nor shared. We propose a more efficient solution for distributed classification by making use of a lossy compression approach applied when sharing the local classifiers with other agents. This approach originates from the framework of hyperdimensional computing, and is adapted herein. The results of experiments on a collection of datasets demonstrate that the proposed approach has usually higher accuracy than local classifiers and getting close to the benchmark - the centralized classifier. This work can be considered as the first step towards analyzing the variegated horizon of distributed randomized neural networks.