SpikeRL: A Scalable and Energy-efficient Framework for Deep Spiking Reinforcement Learning
On the map: Deep reinforcement learning methods → Spiking and neuromorphic RL
Abstract (arXiv)
In this era of AI revolution, massive investments in large-scale data-driven AI systems demand high-performance computing, consuming tremendous energy and resources. This trend raises new challenges in optimizing sustainability without sacrificing scalability or performance. Among the energy-efficient alternatives of the traditional Von Neumann architecture, neuromorphic computing and its Spiking Neural Networks (SNNs) are a promising choice due to their inherent energy efficiency. However, in some real-world application scenarios such as complex continuous control tasks, SNNs often lack the performance optimizations that traditional artificial neural networks have. Researchers have addressed this by combining SNNs with Deep Reinforcement Learning (DeepRL), yet scalability remains unexplored. In this paper, we extend our previous work on SpikeRL, which is a scalable and energy efficient framework for DeepRL-based SNNs for continuous control. In our initial implementation of SpikeRL framework, we depended on the population encoding from the Population-coded Spiking Actor Network (PopSAN) method for our SNN model and implemented distributed training with Message Passing Interface (MPI) through mpi4py. Also, further optimizing our model training by using mixed-precision for parameter updates. In our new SpikeRL framework, we have implemented our own DeepRL-SNN component with population encoding, and distributed training with PyTorch Distributed package with NCCL backend while still optimizing with mixed precision training. Our new SpikeRL implementation is 4.26X faster and 2.25X more energy efficient than state-of-the-art DeepRL-SNN methods. Our proposed SpikeRL framework demonstrates a truly scalable and sustainable solution for complex continuous control tasks in real-world applications.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Spiking and neuromorphic RL, one of 15 directions in Deep reinforcement learning methods. Builds on 8 of its 31 references that are in the map. First posted on arXiv 21 Feb 2025.
Builds on
8 of 31 references.
References · 8
- 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
- 2017Mixed Precision Trainingcited by 1,156
- 2017A Survey of Neuromorphic Computing and Neural Networks in Hardwarecited by 85
- 2020Deep Reinforcement Learning with Population-Coded Spiking Neural Network for Continuous Controlcited by 49
- 2020Reinforcement co-Learning of Deep and Spiking Neural Networks for Energy-Efficient Mapless Navigation with Neuromorphic Hardwarecited by 36
- 2020Neuromorphic control for optic-flow-based landings of MAVs using the Loihi processorcited by 13
- 2018Gridbot: An autonomous robot controlled by a Spiking Neural Network mimicking the brain's navigational systemcited by 7
- 2021A Spiking Central Pattern Generator for the control of a simulated lamprey robot running on SpiNNaker and Loihi neuromorphic boardscited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Spiking Neural Networks for Continuous Control: Neuromorphic Reinforcement Learning in Conventional Computing
- 2025Adaptive Surrogate Gradients for Sequential Reinforcement Learning in Spiking Neural Networks
- 2025Proxy Target: Bridging the Gap Between Discrete Spiking Neural Networks and Continuous Control
- 2024Fully Spiking Actor Network with Intra-layer Connections for Reinforcement Learning
- 2025SINRL: Socially Integrated Navigation with Reinforcement Learning using Spiking Neural Networks
- 2025Energy-Efficient Deep Reinforcement Learning with Spiking Transformers
- 2025Population-Coded Spiking Neural Networks for High-Dimensional Robotic Control
- 2025A PyTorch-Compatible Spike Encoding Framework for Energy-Efficient Neuromorphic Applications