# Strategy and Benchmark for Converting Deep Q-Networks to Event-Driven Spiking Neural Networks

arXiv: https://arxiv.org/abs/2009.14456 · HTML: https://mlsanity.com/paper/2009.14456/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Sep 2020.

## Built on by (most cited first, 23 of 23)

- [Efficient and Accurate Conversion of Spiking Neural Network with Burst Spikes](https://mlsanity.com/paper/2204.13271/) (2022)
- [Human-Level Control through Directly-Trained Deep Spiking Q-Networks](https://mlsanity.com/paper/2201.07211/) (2021)
- [Deep Reinforcement Learning with Spiking Q-learning](https://mlsanity.com/paper/2201.09754/) (2022)
- [One Timestep is All You Need: Training Spiking Neural Networks with Ultra Low Latency](https://mlsanity.com/paper/2110.05929/) (2021)
- [Fully Spiking Actor Network with Intra-layer Connections for Reinforcement Learning](https://mlsanity.com/paper/2401.05444/) (2024)
- [Solving the Spike Feature Information Vanishing Problem in Spiking Deep Q Network with Potential Based Normalization](https://mlsanity.com/paper/2206.03654/) (2022)
- [SpikeCV: Open a Continuous Computer Vision Era](https://mlsanity.com/paper/2303.11684/) (2023)
- [Proxy Target: Bridging the Gap Between Discrete Spiking Neural Networks and Continuous Control](https://mlsanity.com/paper/2505.24161/) (2025)
- [A Low Latency Adaptive Coding Spiking Framework for Deep Reinforcement Learning](https://mlsanity.com/paper/2211.11760/) (2022)
- [Tuning Synaptic Connections instead of Weights by Genetic Algorithm in Spiking Policy Network](https://mlsanity.com/paper/2301.10292/) (2022)
- [Multi-compartment Neuron and Population Encoding Powered Spiking Neural Network for Deep Distributional Reinforcement Learning](https://mlsanity.com/paper/2301.07275/) (2023)
- [GRSN: Gated Recurrent Spiking Neurons for POMDPs and MARL](https://mlsanity.com/paper/2404.15597/) (2024)
- [Improving Performance of Spike-based Deep Q-Learning using Ternary Neurons](https://mlsanity.com/paper/2506.03392/) (2025)
- [CaRe-BN: Precise Moving Statistics for Stabilizing Spiking Neural Networks in Reinforcement Learning](https://mlsanity.com/paper/2509.23791/) (2025)
- [Biologically-Plausible Topology Improved Spiking Actor Network for Efficient Deep Reinforcement Learning](https://mlsanity.com/paper/2403.20163/) (2024)
- [Population-coding and Dynamic-neurons improved Spiking Actor Network for Reinforcement Learning](https://mlsanity.com/paper/2106.07854/) (2021)
- [Decision SpikeFormer: Spike-Driven Transformer for Decision Making](https://mlsanity.com/paper/2504.03800/) (2025)
- [Error Amplification Limits ANN-to-SNN Conversion in Continuous Control](https://mlsanity.com/paper/2601.21778/) (2026)
- [Energy-Efficient Visual Search by Eye Movement and Low-Latency Spiking Neural Network](https://mlsanity.com/paper/2310.06578/) (2023)
- [Robust Iterative Value Conversion: Deep Reinforcement Learning for Neurochip-driven Edge Robots](https://mlsanity.com/paper/2408.13018/) (2024)
- [Fuzzy Encoding-Decoding to Improve Spiking Q-Learning Performance in Autonomous Driving](https://mlsanity.com/paper/2604.16436/) (2026)
- [A Neuromorphic Reinforcement Learning Framework for Efficient Pathfinding in Robotic Mobile Fulfillment Systems](https://mlsanity.com/paper/2606.20031/) (2026)
- [SpikeCredit: Temporal Credit Carrier for Reinforcement Learning with Sparse Rewards](https://mlsanity.com/paper/2609.35268/) (2026)

## Abstract (arXiv)

Spiking neural networks (SNNs) have great potential for energy-efficient implementation of Deep Neural Networks (DNNs) on dedicated neuromorphic hardware. Recent studies demonstrated competitive performance of SNNs compared with DNNs on image classification tasks, including CIFAR-10 and ImageNet data. The present work focuses on using SNNs in combination with deep reinforcement learning in ATARI games, which involves additional complexity as compared to image classification. We review the theory of converting DNNs to SNNs and extending the conversion to Deep Q-Networks (DQNs). We propose a robust representation of the firing rate to reduce the error during the conversion process. In addition, we introduce a new metric to evaluate the conversion process by comparing the decisions made by the DQN and SNN, respectively. We also analyze how the simulation time and parameter normalization influence the performance of converted SNNs. We achieve competitive scores on 17 top-performing Atari games. To the best of our knowledge, our work is the first to achieve state-of-the-art performance on multiple Atari games with SNNs. Our work serves as a benchmark for the conversion of DQNs to SNNs and paves the way for further research on solving reinforcement learning tasks with SNNs.