# HD-CB: The First Exploration of Hyperdimensional Computing for Contextual Bandits Problems

arXiv: https://arxiv.org/abs/2501.16863 · HTML: https://mlsanity.com/paper/2501.16863/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning.
- Builds on 15 of its 54 references that are in the map.
- First posted on arXiv 28 Jan 2025.

## Placed in

- LLM reasoning / [Reinforcement learning for reasoning](https://mlsanity.com/lr/reinforcement-learning-for-reasoning/)

## Builds on (most cited first, 15 of 15)

- [A Survey on Hyperdimensional Computing aka Vector Symbolic Architectures, Part II: Applications, Cognitive Models, and Challenges](https://mlsanity.com/paper/2112.15424/) (2021)
- [A comparison of Vector Symbolic Architectures](https://mlsanity.com/paper/2001.11797/) (2020)
- [Classification using Hyperdimensional Computing: A Review](https://mlsanity.com/paper/2004.11204/) (2020)
- [Vector Symbolic Architectures as a Computing Framework for Emerging Hardware](https://mlsanity.com/paper/2106.05268/) (2021)
- [Holographic Graph Neuron: a Bio-Inspired Architecture for Pattern Processing](https://mlsanity.com/paper/1501.03784/) (2015)
- [Learning from Hypervectors: A Survey on Hypervector Encoding](https://mlsanity.com/paper/2308.00685/) (2023)
- [Customized Nonlinear Bandits for Online Response Selection in Neural Conversation Models](https://mlsanity.com/paper/1711.08493/) (2017)
- [Contextual-Bandit Anomaly Detection for IoT Data in Distributed Hierarchical Edge Computing](https://mlsanity.com/paper/2004.06896/) (2020)
- [One-at-a-time: A Meta-Learning Recommender-System for Recommendation-Algorithm Selection on Micro Level](https://mlsanity.com/paper/1805.12118/) (2018)
- [Edge Computing in the Dark: Leveraging Contextual-Combinatorial Bandit and Coded Computing](https://mlsanity.com/paper/1912.09512/) (2019)
- [Gluing Neural Networks Symbolically Through Hyperdimensional Computing](https://mlsanity.com/paper/2205.15534/) (2022)
- [Efficient Implementation of LinearUCB through Algorithmic Improvements and Vector Computing Acceleration for Embedded Learning Systems](https://mlsanity.com/paper/2501.13139/) (2025)
- [Klessydra-T: Designing Vector Coprocessors for Multi-Threaded Edge-Computing Cores](https://mlsanity.com/paper/2007.09109/) (2020)
- [Sobol Sequence Optimization for Hardware-Efficient Vector Symbolic Architectures](https://mlsanity.com/paper/2311.10277/) (2023)
- [On Effects of Compression with Hyperdimensional Computing in Distributed Randomized Neural Networks](https://mlsanity.com/paper/2106.09831/) (2021)

## Built on by (most cited first, 2 of 2)

- [Large-Margin Hyperdimensional Computing: A Learning-Theoretical Perspective](https://mlsanity.com/paper/2603.03830/) (2026)
- [Contextual Bandits for Resource-Constrained Devices using Probabilistic Learning](https://mlsanity.com/paper/2605.13346/) (2026)

## Abstract (arXiv)

Hyperdimensional Computing (HDC), also known as Vector Symbolic Architectures, is a computing paradigm that combines the strengths of symbolic reasoning with the efficiency and scalability of distributed connectionist models in artificial intelligence. HDC has recently emerged as a promising alternative for performing learning tasks in resource-constrained environments thanks to its energy and computational efficiency, inherent parallelism, and resilience to noise and hardware faults. This work introduces the Hyperdimensional Contextual Bandits (HD-CB): the first exploration of HDC to model and automate sequential decision-making Contextual Bandits (CB) problems. The proposed approach maps environmental states in a high-dimensional space and represents each action with dedicated hypervectors (HVs). At each iteration, these HVs are used to select the optimal action for the given context and are updated based on the received reward, replacing computationally expensive ridge regression procedures required by traditional linear CB algorithms with simple, highly parallel vector operations. We propose four HD-CB variants, demonstrating their flexibility in implementing different exploration strategies, as well as techniques to reduce memory overhead and the number of hyperparameters. Extensive simulations on synthetic datasets and a real-world benchmark reveal that HD-CB consistently achieves competitive or superior performance compared to traditional linear CB algorithms, while offering faster convergence time, lower computational complexity, improved scalability, and high parallelism.