# Implementing Online Reinforcement Learning with Temporal Neural Networks

arXiv: https://arxiv.org/abs/2204.05437 · HTML: https://mlsanity.com/paper/2204.05437/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Apr 2022.

## Built on by (most cited first, 2 of 2)

- [A Macrocolumn Architecture Implemented with Spiking Neurons](https://mlsanity.com/paper/2207.05081/) (2022)
- [Neuromorphic Online Clustering and Its Application to Spike Sorting](https://mlsanity.com/paper/2506.12555/) (2025)

## Abstract (arXiv)

A Temporal Neural Network (TNN) architecture for implementing efficient online reinforcement learning is proposed and studied via simulation. The proposed T-learning system is composed of a frontend TNN that implements online unsupervised clustering and a backend TNN that implements online reinforcement learning. The reinforcement learning paradigm employs biologically plausible neo-Hebbian three-factor learning rules. As a working example, a prototype implementation of the cart-pole problem (balancing an inverted pendulum) is studied via simulation.