mlsanityData to 14 Oct 2026

Biologically-Plausible Topology Improved Spiking Actor Network for Efficient Deep Reinforcement Learning

arXiv 2403.20163 ↗First posted 29 Mar 2024cs.NE q-bio.NCcited by 2

On the map: Deep reinforcement learning methods → Spiking and neuromorphic RL

Abstract (arXiv)

The success of Deep Reinforcement Learning (DRL) is largely attributed to utilizing Artificial Neural Networks (ANNs) as function approximators. Recent advances in neuroscience have unveiled that the human brain achieves efficient reward-based learning, at least by integrating spiking neurons with spatial-temporal dynamics and network topologies with biologically-plausible connectivity patterns. This integration process allows spiking neurons to efficiently combine information across and within layers via nonlinear dendritic trees and lateral interactions. The fusion of these two topologies enhances the network's information-processing ability, crucial for grasping intricate perceptions and guiding decision-making procedures. However, ANNs and brain networks differ significantly. ANNs lack intricate dynamical neurons and only feature inter-layer connections, typically achieved by direct linear summation, without intra-layer connections. This limitation leads to constrained network expressivity. To address this, we propose a novel alternative for function approximator, the Biologically-Plausible Topology improved Spiking Actor Network (BPT-SAN), tailored for efficient decision-making in DRL. The BPT-SAN incorporates spiking neurons with intricate spatial-temporal dynamics and introduces intra-layer connections, enhancing spatial-temporal state representation and facilitating more precise biological simulations. Diverging from the conventional direct linear weighted sum, the BPT-SAN models the local nonlinearities of dendritic trees within the inter-layer connections. For the intra-layer connections, the BPT-SAN introduces lateral interactions between adjacent neurons, integrating them into the membrane potential formula to ensure accurate spike firing.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Spiking and neuromorphic RL, one of 15 directions in Deep reinforcement learning methods. Builds on 19 of its 45 references that are in the map. First posted on arXiv 29 Mar 2024.

Builds on

19 of 45 references.

References · 19

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2015Continuous control with deep reinforcement learningcited by 3,407
  4. 2015Trust Region Policy Optimizationcited by 2,580
  5. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  6. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  7. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
  8. 2016OpenAI Gymcited by 1,736
  9. 2015Dueling Network Architectures for Deep Reinforcement Learningcited by 785
  10. 2017Deep Reinforcement Learning that Matterscited by 754
  11. 2016Benchmarking Deep Reinforcement Learning for Continuous Controlcited by 371
  12. 2017SuperSpike: Supervised learning in multi-layer spiking neural networkscited by 182
7 more
  1. 2017A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problemcited by 128
  2. 2020Deep Reinforcement Learning with Population-Coded Spiking Neural Network for Continuous Controlcited by 49
  3. 2020Reinforcement co-Learning of Deep and Spiking Neural Networks for Energy-Efficient Mapless Navigation with Neuromorphic Hardwarecited by 36
  4. 2020Strategy and Benchmark for Converting Deep Q-Networks to Event-Driven Spiking Neural Networkscited by 23
  5. 2023ODE-based Recurrent Model-free Reinforcement Learning for POMDPscited by 7
  6. 2022Tuning Synaptic Connections instead of Weights by Genetic Algorithm in Spiking Policy Networkcited by 4
  7. 2023Attention-free Spikformer: Mixing Spike Sequences with Simple Linear Transformscited by 3

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.

  1. 2025Proxy Target: Bridging the Gap Between Discrete Spiking Neural Networks and Continuous Controlcited by 6
  2. 2025CaRe-BN: Precise Moving Statistics for Stabilizing Spiking Neural Networks in Reinforcement Learningcited by 3

Similar, not cited

Close in wording, no citation either way

  1. 2025Spiking World Model with Multi-Compartment Neurons for Model-based Reinforcement Learning
  2. 2025Energy-Efficient Deep Reinforcement Learning with Spiking Transformers
  3. 2025Spiking Decision Transformers: Local Plasticity, Phase-Coding, and Dendritic Routing for Low-Power Sequence Control
  4. 2024Fully Spiking Actor Network with Intra-layer Connections for Reinforcement Learning
  5. 2023Biologically inspired structure learning with reverse knowledge distillation for spiking neural networks
  6. 2025SpikeRL: A Scalable and Energy-efficient Framework for Deep Spiking Reinforcement Learning
  7. 2025Intelligent Neural Networks: From Layered Architectures to Graph-Organized Intelligence
  8. 2024GRSN: Gated Recurrent Spiking Neurons for POMDPs and MARL