# Efficient Off-Policy Reinforcement Learning via Brain-Inspired Computing

arXiv: https://arxiv.org/abs/2205.06978 · HTML: https://mlsanity.com/paper/2205.06978/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 19 references that are in the map.
- First posted on arXiv 14 May 2022.

## Builds on (most cited first, 3 of 3)

- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [Optimizing Online Matching for Ride-Sourcing Services with Multi-Agent Deep Reinforcement Learning](https://mlsanity.com/paper/1902.06228/) (2019)

## Built on by (most cited first, 2 of 2)

- [Analogical Reasoning Within a Conceptual Hyperspace](https://mlsanity.com/paper/2411.08684/) (2024)
- [Reinforcement Learning Enhancement Using Vector Semantic Representation and Symbolic Reasoning for Human-Centered Autonomous Emergency Braking](https://mlsanity.com/paper/2602.05079/) (2026)

## Abstract (arXiv)

Reinforcement Learning (RL) has opened up new opportunities to enhance existing smart systems that generally include a complex decision-making process. However, modern RL algorithms, e.g., Deep Q-Networks (DQN), are based on deep neural networks, resulting in high computational costs. In this paper, we propose QHD, an off-policy value-based Hyperdimensional Reinforcement Learning, that mimics brain properties toward robust and real-time learning. QHD relies on a lightweight brain-inspired model to learn an optimal policy in an unknown environment. On both desktop and power-limited embedded platforms, QHD achieves significantly better overall efficiency than DQN while providing higher or comparable rewards. QHD is also suitable for highly-efficient reinforcement learning with great potential for online and real-time learning. Our solution supports a small experience replay batch size that provides 12.3 times speedup compared to DQN while ensuring minimal quality loss. Our evaluation shows QHD capability for real-time learning, providing 34.6 times speedup and significantly better quality of learning than DQN.