mlsanityData to 14 Oct 2026

Time to Spike? Understanding the Representational Power of Spiking Neural Networks in Discrete Time

arXiv 2505.18023 ↗First posted 23 May 2025cs.LG cs.NEcited by 4

Not on the map. Why?

Abstract (arXiv)

Recent years have seen significant progress in developing spiking neural networks (SNNs) as a potential solution to the energy challenges posed by conventional artificial neural networks (ANNs). However, our theoretical understanding of SNNs remains relatively limited compared to the ever-growing body of literature on ANNs. In this paper, we study a discrete-time model of SNNs based on leaky integrate-and-fire (LIF) neurons, referred to as discrete-time LIF-SNNs, a widely used framework that still lacks solid theoretical foundations. We demonstrate that discrete-time LIF-SNNs with static inputs and outputs realize piecewise constant functions defined on polyhedral regions, and more importantly, we quantify the network size required to approximate continuous functions. Moreover, we investigate the impact of latency (number of time steps) and depth (number of layers) on the complexity of the input space partitioning induced by discrete-time LIF-SNNs. Our analysis highlights the importance of latency and contrasts these networks with ANNs employing piecewise linear activation functions. Finally, we present numerical experiments to support our theoretical findings.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 44 of its 75 references that are in the map. First posted on arXiv 23 May 2025.

Builds on

44 of 75 references.

References · 44

  1. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,611
  2. 2019Surrogate Gradient Learning in Spiking Neural Networkscited by 626
  3. 2016Error bounds for approximations with deep ReLU networkscited by 574
  4. 2014On the Number of Linear Regions of Deep Neural Networkscited by 381
  5. 2020Incorporating Learnable Membrane Time Constant to Enhance Learning of Spiking Neural Networkscited by 345
  6. 2017The Expressive Power of Neural Networks: A View from the Widthcited by 320
  7. 2016On the Expressive Power of Deep Neural Networkscited by 306
  8. 2021Training Spiking Neural Networks Using Lessons From Deep Learningcited by 259
  9. 2016Understanding Deep Neural Networks with Rectified Linear Unitscited by 257
  10. 2016Benefits of depth in neural networkscited by 250
  11. 2018Direct Training for Spiking Neural Networks: Faster, Larger, Bettercited by 233
  12. 2016Training Deep Spiking Neural Networks using Backpropagationcited by 205
32 more
  1. 2017Optimal approximation of piecewise smooth functions using deep ReLU neural networkscited by 179
  2. 2018Universality of Deep Convolutional Neural Networkscited by 176
  3. 2023SpikingJelly: An open-source machine learning infrastructure platform for spike-based intelligencecited by 169
  4. 2019Complexity of Linear Regions in Deep Networkscited by 167
  5. 2017Bounding and Counting Linear Regions of Deep Neural Networkscited by 161
  6. 2019Deep ReLU Networks Have Surprisingly Few Activation Patternscited by 161
  7. 2019Deep Network Approximation Characterized by Number of Neuronscited by 122
  8. 2017Universal Function Approximation by Deep Neural Nets with Bounded Width and ReLU Activationscited by 104
  9. 2021Efficient Neuromorphic Signal Processing with Loihi 2cited by 96
  10. 2019Error bounds for approximations with deep ReLU neural networks in Ws,p normscited by 95
  11. 2017Optimal Approximation with Sparsely Connected Deep Neural Networkscited by 91
  12. 2016Supervised learning based on temporal coding in spiking neural networkscited by 76
  13. 2019A Theoretical Analysis of Deep Neural Networks and Parametric PDEscited by 74
  14. 2019The Geometry of Deep Networks: Power Diagram Subdivisioncited by 53
  15. 2022Neural Architecture Search for Spiking Neural Networkscited by 53
  16. 2019Fast and energy-efficient neuromorphic deep learning with first-spike timescited by 52
  17. 2024Deep Networks Always Grok and Here is Whycited by 51
  18. 2019Temporal Coding in Spiking Neural Networks with Alpha Synaptic Function: Learning with Backpropagationcited by 45
  19. 2023Parallel Spiking Neurons with High Efficiency and Ability to Learn Long-term Dependenciescited by 41
  20. 2023When Deep Learning Meets Polyhedral Theory: A Surveycited by 39
  21. 2022An Analytical Estimation of Spiking Neural Networks Energy Efficiencycited by 38
  22. 2020On transversality of bent hyperplane arrangements and the topological expressiveness of ReLU neural networkscited by 35
  23. 2023Direct Learning-Based Deep Spiking Neural Networks: A Reviewcited by 31
  24. 2023High-performance deep spiking neural networks with 0.3 spikes per neuroncited by 22
  25. 2021Neural networks with linear threshold activations: structure and algorithmscited by 12
  26. 2024Efficient and Effective Time-Series Forecasting with Spiking Neural Networkscited by 12
  27. 2024Roadmap to Neuromorphic Computing with Emerging Technologiescited by 12
  28. 2022Spiking neural networks for nonlinear regressioncited by 10
  29. 2024On the Local Complexity of Linear Regions in Deep ReLU Networkscited by 9
  30. 2024Stable Learning Using Spiking Neural Networks Equipped With Affine Encoders and Decoderscited by 5
  31. 2022On the Intrinsic Structures of Spiking Neural Networkscited by 4
  32. 2017On Separating Points by Linescited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026AffineLens: Capturing the Continuous Piecewise Affine Functions of Neural Networkscited by 1
  2. 2026Complexity of Linear Regions in Self-supervised Deep ReLU Networkscited by 0
  3. 2026Expressivity Saturation: Reduced Affine Region Usage Under Increasing Task Complexitycited by 0
  4. 2026Towards an Expressivity-Normalized Energy-Demand Comparison of ANNs and SNNscited by 0