# Implementing Online Reinforcement Learning with Clustering Neural Networks

arXiv: https://arxiv.org/abs/2402.18472 · HTML: https://mlsanity.com/paper/2402.18472/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 5 of its 15 references that are in the map.
- First posted on arXiv 28 Feb 2024.

## Placed in

- Deep reinforcement learning methods / [Deep RL algorithms and representations](https://mlsanity.com/deeprl/deep-rl-algorithms-and-representations/)

## Builds on (most cited first, 5 of 5)

- [Eligibility Traces and Plasticity on Behavioral Time Scales: Experimental Support of neoHebbian Three-Factor Learning Rules](https://mlsanity.com/paper/1801.05219/) (2018)
- [Why Neurons Have Thousands of Synapses, A Theory of Sequence Memory in Neocortex](https://mlsanity.com/paper/1511.00083/) (2015)
- [Comparison of Reinforcement Learning algorithms applied to the Cart Pole problem](https://mlsanity.com/paper/1810.01940/) (2018)
- [Encoding Data for HTM Systems](https://mlsanity.com/paper/1602.05925/) (2016)
- [Neuromorphic Online Clustering and Classification](https://mlsanity.com/paper/2310.17797/) (2023)

## Abstract (arXiv)

An agent employing reinforcement learning takes inputs (state variables) from an environment and performs actions that affect the environment in order to achieve some objective. Rewards (positive or negative) guide the agent toward improved future actions. This paper builds on prior clustering neural network research by constructing an agent with biologically plausible neo-Hebbian three-factor synaptic learning rules, with a reward signal as the third factor (in addition to pre- and post-synaptic spikes). The classic cart-pole problem (balancing an inverted pendulum) is used as a running example throughout the exposition. Simulation results demonstrate the efficacy of the approach, and the proposed method may eventually serve as a low-level component of a more general method.