mlsanityData to 14 Oct 2026

Common-Mode Errors Limit Low-Timestep Deep Spiking Q-Networks

arXiv 2610.07808 ↗First posted 6 Oct 2026cs.NE cs.LGcited by 0

On the map: Deep reinforcement learning methods → Spiking and neuromorphic RL

Abstract (arXiv)

Spiking neural networks (SNNs) offer sparse and event-driven computation, making them attractive for energy-constrained reinforcement learning (RL) on edge devices. In value-based RL, deep spiking Q-networks (DSQNs) combine such efficiency with action-value estimation for decision making. However, existing DSQNs often require multiple simulation timesteps for competitive performance, increasing computational and energy costs, whereas reducing the timesteps can cause substantial performance degradation. We investigate this degradation from the perspective of Q-value estimation errors. By decomposing errors across actions into common-mode and differential-mode components, we find that low-timestep DSQNs suffer disproportionately from common-mode errors shared across action values, which are particularly detrimental to temporal-difference learning through bootstrapped targets. Based on this finding, we propose Common-Mode Compensation Deep Spiking Q-Network (CMC-DSQN), which uses an auxiliary ANN to compensate for common-mode errors in the SNN outputs. At inference, greedy action selection can be performed directly from the SNN outputs, allowing the auxiliary ANN to be completely removed and preserving the energy efficiency of SNNs. Extensive experiments on Atari and MiniAtar environments demonstrate substantial performance improvements under low-timestep settings. CMC-DSQN outperforms state-of-the-art DSQN baselines by nearly 20% at T=2 and further surpasses the ANN baseline at T=4.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Spiking and neuromorphic RL, one of 15 directions in Deep reinforcement learning methods. Builds on 9 of its 50 references that are in the map. First posted on arXiv 6 Oct 2026.

Builds on

9 of 50 references.

References · 9

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2015Continuous control with deep reinforcement learningcited by 3,407
  3. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  4. 2015Prioritized Experience Replaycited by 1,099
  5. 2019MinAtar: An Atari-Inspired Testbed for Thorough and Reproducible Reinforcement Learning Experimentscited by 19
  6. 2022Deep Reinforcement Learning with Spiking Q-learningcited by 18
  7. 2025Differential Coding for Training-Free ANN-to-SNN Conversioncited by 10
  8. 2022A Low Latency Adaptive Coding Spiking Framework for Deep Reinforcement Learningcited by 5
  9. 2026Error Amplification Limits ANN-to-SNN Conversion in Continuous Controlcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Improving Performance of Spike-based Deep Q-Learning using Ternary Neurons
  2. 2021Q-SpiNN: A Framework for Quantizing Spiking Neural Networks
  3. 2025Energy-Efficient and Dequantization-Free Q-LLMs: A Spiking Neural Network Approach to Salient Value Mitigation
  4. 2025Learning to Control Dynamical Agents via Spiking Neural Networks and Metropolis-Hastings Sampling
  5. 2026Latency and accuracy tradeoffs in Spiking Neural Networks
  6. 2026SPARQ: Spiking Early-Exit Neural Networks for Energy-Efficient Edge AI
  7. 2024Coarse-to-fine Q-Network with Action Sequence for Data-Efficient Reinforcement Learning
  8. 2024GRSN: Gated Recurrent Spiking Neurons for POMDPs and MARL