mlsanityData to 14 Oct 2026

Solving the Spike Feature Information Vanishing Problem in Spiking Deep Q Network with Potential Based Normalization

arXiv 2206.03654 ↗First posted 8 Jun 2022cs.NE cs.AI cs.LGcited by 10

Not on the map. Why?

Abstract (arXiv)

Brain inspired spiking neural networks (SNNs) have been successfully applied to many pattern recognition domains. The SNNs based deep structure have achieved considerable results in perceptual tasks, such as image classification, target detection. However, the application of deep SNNs in reinforcement learning (RL) tasks is still a problem to be explored. Although there have been previous studies on the combination of SNNs and RL, most of them focus on robotic control problems with shallow networks or using ANN-SNN conversion method to implement spiking deep Q Network (SDQN). In this work, we mathematically analyzed the problem of the disappearance of spiking signal features in SDQN and proposed a potential based layer normalization(pbLN) method to directly train spiking deep Q networks. Experiment shows that compared with state-of-art ANN-SNN conversion method and other SDQN works, the proposed pbLN spiking deep Q networks (PL-SDQN) achieved better performance on Atari game tasks.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 14 of its 32 references that are in the map. First posted on arXiv 8 Jun 2022.

Builds on

14 of 32 references.

References · 14

  1. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  2. 2017Spatio-Temporal Backpropagation for Training High-performance Spiking Neural Networkscited by 406
  3. 2021Deep Residual Learning in Spiking Neural Networkscited by 331
  4. 2020Going Deeper With Directly-Trained Larger Spiking Neural Networkscited by 278
  5. 2018Direct Training for Spiking Neural Networks: Faster, Larger, Bettercited by 233
  6. 2019Spiking-YOLO: Spiking Neural Network for Energy-Efficient Object Detectioncited by 177
  7. 2020Deep Reinforcement Learning with Population-Coded Spiking Neural Network for Continuous Controlcited by 49
  8. 2022Efficient and Accurate Conversion of Spiking Neural Network with Burst Spikescited by 30
  9. 2021Human-Level Control through Directly-Trained Deep Spiking Q-Networkscited by 25
  10. 2020Strategy and Benchmark for Converting Deep Q-Networks to Event-Driven Spiking Neural Networkscited by 23
  11. 2022Deep Reinforcement Learning with Spiking Q-learningcited by 18
  12. 2020Learning to Walk: Spike Based Reinforcement Learning for Hexapod Robot Central Pattern Generationcited by 6
2 more
  1. 2021Distilling Neuron Spike with High Temperature in Reinforcement Learning Agentscited by 5
  2. 2020Quantum Superposition Inspired Spiking Neural Networkcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 3; 2025: 3; 2026: 2.

10 papers.

  1. 2024Fully Spiking Actor Network with Intra-layer Connections for Reinforcement Learningcited by 11
  2. 2025Proxy Target: Bridging the Gap Between Discrete Spiking Neural Networks and Continuous Controlcited by 6
  3. 2023Multi-compartment Neuron and Population Encoding Powered Spiking Neural Network for Deep Distributional Reinforcement Learningcited by 3
  4. 2024GRSN: Gated Recurrent Spiking Neurons for POMDPs and MARLcited by 3
  5. 2025Improving Performance of Spike-based Deep Q-Learning using Ternary Neuronscited by 3
  6. 2025CaRe-BN: Precise Moving Statistics for Stabilizing Spiking Neural Networks in Reinforcement Learningcited by 3
  7. 2023S3TC: Spiking Separated Spatial and Temporal Convolutions with Unsupervised STDP-based Learning for Action Recognitioncited by 1
  8. 2024Brain-inspired and Self-based Artificial Intelligencecited by 1
  9. 2026Error Amplification Limits ANN-to-SNN Conversion in Continuous Controlcited by 1
  10. 2026Fuzzy Encoding-Decoding to Improve Spiking Q-Learning Performance in Autonomous Drivingcited by 0