# Solving the Spike Feature Information Vanishing Problem in Spiking Deep Q Network with Potential Based Normalization

arXiv: https://arxiv.org/abs/2206.03654 · HTML: https://mlsanity.com/paper/2206.03654/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 32 references that are in the map.
- First posted on arXiv 8 Jun 2022.

## Builds on (most cited first, 14 of 14)

- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Spatio-Temporal Backpropagation for Training High-performance Spiking Neural Networks](https://mlsanity.com/paper/1706.02609/) (2017)
- [Deep Residual Learning in Spiking Neural Networks](https://mlsanity.com/paper/2102.04159/) (2021)
- [Going Deeper With Directly-Trained Larger Spiking Neural Networks](https://mlsanity.com/paper/2011.05280/) (2020)
- [Direct Training for Spiking Neural Networks: Faster, Larger, Better](https://mlsanity.com/paper/1809.05793/) (2018)
- [Spiking-YOLO: Spiking Neural Network for Energy-Efficient Object Detection](https://mlsanity.com/paper/1903.06530/) (2019)
- [Deep Reinforcement Learning with Population-Coded Spiking Neural Network for Continuous Control](https://mlsanity.com/paper/2010.09635/) (2020)
- [Efficient and Accurate Conversion of Spiking Neural Network with Burst Spikes](https://mlsanity.com/paper/2204.13271/) (2022)
- [Human-Level Control through Directly-Trained Deep Spiking Q-Networks](https://mlsanity.com/paper/2201.07211/) (2021)
- [Strategy and Benchmark for Converting Deep Q-Networks to Event-Driven Spiking Neural Networks](https://mlsanity.com/paper/2009.14456/) (2020)
- [Deep Reinforcement Learning with Spiking Q-learning](https://mlsanity.com/paper/2201.09754/) (2022)
- [Learning to Walk: Spike Based Reinforcement Learning for Hexapod Robot Central Pattern Generation](https://mlsanity.com/paper/2003.10026/) (2020)
- [Distilling Neuron Spike with High Temperature in Reinforcement Learning Agents](https://mlsanity.com/paper/2108.10078/) (2021)
- [Quantum Superposition Inspired Spiking Neural Network](https://mlsanity.com/paper/2010.12197/) (2020)

## Built on by (most cited first, 10 of 10)

- [Fully Spiking Actor Network with Intra-layer Connections for Reinforcement Learning](https://mlsanity.com/paper/2401.05444/) (2024)
- [Proxy Target: Bridging the Gap Between Discrete Spiking Neural Networks and Continuous Control](https://mlsanity.com/paper/2505.24161/) (2025)
- [Multi-compartment Neuron and Population Encoding Powered Spiking Neural Network for Deep Distributional Reinforcement Learning](https://mlsanity.com/paper/2301.07275/) (2023)
- [GRSN: Gated Recurrent Spiking Neurons for POMDPs and MARL](https://mlsanity.com/paper/2404.15597/) (2024)
- [Improving Performance of Spike-based Deep Q-Learning using Ternary Neurons](https://mlsanity.com/paper/2506.03392/) (2025)
- [CaRe-BN: Precise Moving Statistics for Stabilizing Spiking Neural Networks in Reinforcement Learning](https://mlsanity.com/paper/2509.23791/) (2025)
- [S3TC: Spiking Separated Spatial and Temporal Convolutions with Unsupervised STDP-based Learning for Action Recognition](https://mlsanity.com/paper/2309.12761/) (2023)
- [Brain-inspired and Self-based Artificial Intelligence](https://mlsanity.com/paper/2402.18784/) (2024)
- [Error Amplification Limits ANN-to-SNN Conversion in Continuous Control](https://mlsanity.com/paper/2601.21778/) (2026)
- [Fuzzy Encoding-Decoding to Improve Spiking Q-Learning Performance in Autonomous Driving](https://mlsanity.com/paper/2604.16436/) (2026)

## Abstract (arXiv)

Brain inspired spiking neural networks (SNNs) have been successfully applied to many pattern recognition domains. The SNNs based deep structure have achieved considerable results in perceptual tasks, such as image classification, target detection. However, the application of deep SNNs in reinforcement learning (RL) tasks is still a problem to be explored. Although there have been previous studies on the combination of SNNs and RL, most of them focus on robotic control problems with shallow networks or using ANN-SNN conversion method to implement spiking deep Q Network (SDQN). In this work, we mathematically analyzed the problem of the disappearance of spiking signal features in SDQN and proposed a potential based layer normalization(pbLN) method to directly train spiking deep Q networks. Experiment shows that compared with state-of-art ANN-SNN conversion method and other SDQN works, the proposed pbLN spiking deep Q networks (PL-SDQN) achieved better performance on Atari game tasks.