# Multi-compartment Neuron and Population Encoding Powered Spiking Neural Network for Deep Distributional Reinforcement Learning

arXiv: https://arxiv.org/abs/2301.07275 · HTML: https://mlsanity.com/paper/2301.07275/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 54 references that are in the map.
- First posted on arXiv 18 Jan 2023.

## Builds on (most cited first, 21 of 21)

- [Spatio-Temporal Backpropagation for Training High-performance Spiking Neural Networks](https://mlsanity.com/paper/1706.02609/) (2017)
- [Spiking-YOLO: Spiking Neural Network for Energy-Efficient Object Detection](https://mlsanity.com/paper/1903.06530/) (2019)
- [Enabling Spike-based Backpropagation for Training Deep Neural Network Architectures](https://mlsanity.com/paper/1903.06379/) (2019)
- [Dendritic cortical microcircuits approximate the backpropagation algorithm](https://mlsanity.com/paper/1810.11393/) (2018)
- [Fully Parameterized Quantile Function for Distributional Reinforcement Learning](https://mlsanity.com/paper/1911.02140/) (2019)
- [Deep Reinforcement Learning with Population-Coded Spiking Neural Network for Continuous Control](https://mlsanity.com/paper/2010.09635/) (2020)
- [Reinforcement co-Learning of Deep and Spiking Neural Networks for Energy-Efficient Mapless Navigation with Neuromorphic Hardware](https://mlsanity.com/paper/2003.01157/) (2020)
- [BrainCog: A Spiking Neural Network based Brain-inspired Cognitive Intelligence Engine for Brain-inspired AI and Brain Simulation](https://mlsanity.com/paper/2207.08533/) (2022)
- [Efficient and Accurate Conversion of Spiking Neural Network with Burst Spikes](https://mlsanity.com/paper/2204.13271/) (2022)
- [Learning to solve the credit assignment problem](https://mlsanity.com/paper/1906.00889/) (2019)
- [Spiking Neural Networks with Improved Inherent Recurrence Dynamics for Sequential Learning](https://mlsanity.com/paper/2109.01905/) (2021)
- [Human-Level Control through Directly-Trained Deep Spiking Q-Networks](https://mlsanity.com/paper/2201.07211/) (2021)
- [Strategy and Benchmark for Converting Deep Q-Networks to Event-Driven Spiking Neural Networks](https://mlsanity.com/paper/2009.14456/) (2020)
- [Deep Reinforcement Learning with Spiking Q-learning](https://mlsanity.com/paper/2201.09754/) (2022)
- [SiamSNN: Siamese Spiking Neural Networks for Energy-Efficient Object Tracking](https://mlsanity.com/paper/2003.07584/) (2020)
- [In-Hardware Learning of Multilayer Spiking Neural Networks on a Neuromorphic Processor](https://mlsanity.com/paper/2105.03649/) (2021)
- [Solving the Spike Feature Information Vanishing Problem in Spiking Deep Q Network with Potential Based Normalization](https://mlsanity.com/paper/2206.03654/) (2022)
- [Distilling Neuron Spike with High Temperature in Reinforcement Learning Agents](https://mlsanity.com/paper/2108.10078/) (2021)
- [Deep Convolutional Spiking Neural Networks for Image Classification](https://mlsanity.com/paper/1903.12272/) (2019)
- [A Spiking Neural Network Emulating the Structure of the Oculomotor System Requires No Learning to Control a Biomimetic Robotic Head](https://mlsanity.com/paper/2002.07534/) (2020)
- [Burst-dependent plasticity and dendritic amplification support target-based learning and hierarchical imitation learning](https://mlsanity.com/paper/2201.11717/) (2022)

## Built on by (most cited first, 3 of 3)

- [Proxy Target: Bridging the Gap Between Discrete Spiking Neural Networks and Continuous Control](https://mlsanity.com/paper/2505.24161/) (2025)
- [Noisy group neurons with synchronous resetting for high-performance spiking neural networks](https://mlsanity.com/paper/2608.17394/) (2026)
- [Spiking Neural Networks for Continuous Control: Neuromorphic Reinforcement Learning in Conventional Computing](https://mlsanity.com/paper/2608.22729/) (2026)

## Abstract (arXiv)

Inspired by the brain's information processing using binary spikes, spiking neural networks (SNNs) offer significant reductions in energy consumption and are more adept at incorporating multi-scale biological characteristics. In SNNs, spiking neurons serve as the fundamental information processing units. However, in most models, these neurons are typically simplified, focusing primarily on the leaky integrate-and-fire (LIF) point neuron model while neglecting the structural properties of biological neurons. This simplification hampers the computational and learning capabilities of SNNs. In this paper, we propose a brain-inspired deep distributional reinforcement learning algorithm based on SNNs, which integrates a bio-inspired multi-compartment neuron (MCN) model with a population coding approach. The proposed MCN model simulates the structure and function of apical dendritic, basal dendritic, and somatic compartments, achieving computational power comparable to that of biological neurons. Additionally, we introduce an implicit fractional embedding method based on population coding of spiking neurons. We evaluated our model on Atari games, and the experimental results demonstrate that it surpasses the vanilla FQF model, which utilizes traditional artificial neural networks (ANNs), as well as the Spiking-FQF models that are based on ANN-to-SNN conversion methods. Ablation studies further reveal that the proposed multi-compartment neuron model and the quantile fraction implicit population spike representation significantly enhance the performance of MCS-FQF while also reducing power consumption.