# Spiking Neural Network Predicting Sequence of the External Worlds States in Model-Based Reinforcement Learning

arXiv: https://arxiv.org/abs/2609.27459 · HTML: https://mlsanity.com/paper/2609.27459/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Spiking and neuromorphic RL, one of 15 directions in Deep reinforcement learning methods.
- Builds on 4 of its 19 references that are in the map.
- First posted on arXiv 23 Sep 2026.

## Placed in

- Deep reinforcement learning methods / [Spiking and neuromorphic RL](https://mlsanity.com/deeprl/spiking-and-neuromorphic-rl/)

## Builds on (most cited first, 4 of 4)

- [Surrogate Gradient Learning in Spiking Neural Networks](https://mlsanity.com/paper/1901.09948/) (2019)
- [Spiking World Model with Multi-Compartment Neurons for Model-based Reinforcement Learning](https://mlsanity.com/paper/2503.00713/) (2025)
- [Continual Learning with Columnar Spiking Neural Networks](https://mlsanity.com/paper/2506.17169/) (2025)
- [A Spiking Neural Network Learning Markov Chain](https://mlsanity.com/paper/2209.09572/) (2022)

## Abstract (arXiv)

This paper presents a spiking neural network (SNN) designed to predict the sequence of the external world states starting from the current world state. This SNN does not create the world dynamics model - instead it incorporates the SNN trained to predict the next world state and provides all mechanisms necessary to make the chain of predicted world states. These mechanisms are entirely spiking - they are implemented as spiking neuron ensembles. The present article describes this neuronal structure and tests its operation on a classic RL benchmark - ATARI ping-pong.