mlsanityData to 14 Oct 2026

An Exact Mapping From ReLU Networks to Spiking Neural Networks

arXiv 2212.12522 ↗First posted 23 Dec 2022cs.NE cs.LGcited by 12

Not on the map. Why?

Abstract (arXiv)

Deep spiking neural networks (SNNs) offer the promise of low-power artificial intelligence. However, training deep SNNs from scratch or converting deep artificial neural networks to SNNs without loss of performance has been a challenge. Here we propose an exact mapping from a network with Rectified Linear Units (ReLUs) to an SNN that fires exactly one spike per neuron. For our constructive proof, we assume that an arbitrary multi-layer ReLU network with or without convolutional layers, batch normalization and max pooling layers was trained to high performance on some training set. Furthermore, we assume that we have access to a representative example of input data used during training and to the exact parameters (weights and biases) of the trained ReLU network. The mapping from deep ReLU networks to SNNs causes zero percent drop in accuracy on CIFAR10, CIFAR100 and the ImageNet-like data sets Places365 and PASS. More generally our work shows that an arbitrary deep ReLU network can be replaced by an energy-efficient single-spike neural network without any loss of performance.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 24 of its 65 references that are in the map. First posted on arXiv 23 Dec 2022.

Builds on

24 of 65 references.

References · 24

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  4. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  5. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  6. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,845
  7. 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,254
  8. 2019Event-based Vision: A Surveycited by 1,016
  9. 2019Surrogate Gradient Learning in Spiking Neural Networkscited by 626
  10. 2018Deep Learning in Spiking Neural Networkscited by 263
  11. 2017SuperSpike: Supervised learning in multi-layer spiking neural networkscited by 182
  12. 2019Convergence of Edge Computing and Deep Learning: A Comprehensive Surveycited by 120
12 more
  1. 2021A Study of Face Obfuscation in ImageNetcited by 83
  2. 2016Supervised learning based on temporal coding in spiking neural networkscited by 76
  3. 2019Fast and energy-efficient neuromorphic deep learning with first-spike timescited by 52
  4. 2021PASS: An ImageNet replacement for self-supervised pretraining without humanscited by 51
  5. 2019S4NN: temporal backpropagation for spiking neural networks with one spike per neuroncited by 49
  6. 2017Gradient Descent for Spiking Neural Networkscited by 48
  7. 2020T2FSNN: Deep Spiking Neural Networks with Time-to-first-spike Codingcited by 46
  8. 2019Temporal Coding in Spiking Neural Networks with Alpha Synaptic Function: Learning with Backpropagationcited by 45
  9. 2016Conversion of Artificial Recurrent Neural Networks to Spiking Neural Networks for Low-power Neuromorphic Hardwarecited by 40
  10. 2019Optimizing the energy consumption of spiking neural networks for neuromorphic applicationscited by 37
  11. 2021Google Neural Network Models for Edge Devices: Analyzing and Mitigating Machine Learning Inference Bottleneckscited by 23
  12. 2015Encoding Spike Patterns in Multilayer Spiking Neural Networkscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 4; 2025: 3; 2026: 3.

12 papers.

  1. 2023High-performance deep spiking neural networks with 0.3 spikes per neuroncited by 22
  2. 2025Differential Coding for Training-Free ANN-to-SNN Conversioncited by 10
  3. 2024Stable Learning Using Spiking Neural Networks Equipped With Affine Encoders and Decoderscited by 5
  4. 2024Sign Gradient Descent-based Neuronal Dynamics: ANN-to-SNN Conversion Beyond ReLU Networkcited by 5
  5. 2023Deep ReLU networks and high-order finite element methods II: Chebyshev emulationcited by 3
  6. 2025Causal pieces: analysing and improving spiking neural networks piece by piececited by 2
  7. 2024Neural Networks for Singular Perturbationscited by 1
  8. 2026Error Amplification Limits ANN-to-SNN Conversion in Continuous Controlcited by 1
  9. 2026Equivalence of approximation by networks of single- and multi-spike neuronscited by 1
  10. 2024Obtaining Optimal Spiking Neural Network in Sequence Learning via CRNN-SNN Conversioncited by 0
  11. 2025All in one timestep: Enhancing Sparsity and Energy efficiency in Multi-level Spiking Neural Networkscited by 0
  12. 2026Large Language Models with At Most One Spike per Neuroncited by 0