mlsanityData to 7 Oct 2026

Recent Advances in Scalable Energy-Efficient and Trustworthy Spiking Neural networks: from Algorithms to Technology

arXiv 2312.01213 ↗First posted 2 Dec 2023cs.AR cs.AIcited by 3

Not on the map. Why?

Abstract (arXiv)

Neuromorphic computing and, in particular, spiking neural networks (SNNs) have become an attractive alternative to deep neural networks for a broad range of signal processing applications, processing static and/or temporal inputs from different sensory modalities, including audio and vision sensors. In this paper, we start with a description of recent advances in algorithmic and optimization innovations to efficiently train and scale low-latency, and energy-efficient spiking neural networks (SNNs) for complex machine learning applications. We then discuss the recent efforts in algorithm-architecture co-design that explores the inherent trade-offs between achieving high energy-efficiency and low latency while still providing high accuracy and trustworthiness. We then describe the underlying hardware that has been developed to leverage such algorithmic innovations in an efficient way. In particular, we describe a hybrid method to integrate significant portions of the model's computation within both memory components as well as the sensor itself. Finally, we discuss the potential path forward for research in building deployable SNN systems identifying key challenges in the algorithm-hardware-application co-design space with an emphasis on trustworthiness.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 43 of its 56 references that are in the map. First posted on arXiv 2 Dec 2023.

Builds on

43 of 56 references.

References · 43

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  5. 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,242
  6. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
  7. 2023RWKV: Reinventing RNNs for the Transformer Eracited by 814
  8. 2023Segment Everything Everywhere All at Oncecited by 546
  9. 2021Deep Residual Learning in Spiking Neural Networkscited by 326
  10. 2020Going Deeper With Directly-Trained Larger Spiking Neural Networkscited by 268
  11. 2022Spikformer: When Spiking Neural Network Meets Transformercited by 263
  12. 2018Going Deeper in Spiking Neural Networks: VGG and Residual Architecturescited by 236
31 more
  1. 2021Mobile-Former: Bridging MobileNet and Transformercited by 234
  2. 2022cosFormer: Rethinking Softmax in Attentioncited by 212
  3. 2018Long short-term memory and learning-to-learn in networks of spiking neuronscited by 179
  4. 2020Enabling Deep Spiking Neural Networks with Hybrid Conversion and Spike Timing Dependent Backpropagationcited by 142
  5. 2023Spike-driven Transformercited by 141
  6. 2021Optimal Conversion of Conventional Artificial Neural Networks to Spiking Neural Networkscited by 134
  7. 2022Attention Spiking Neural Networkscited by 103
  8. 2023SpikeGPT: Generative Pre-trained Language Model with Spiking Neural Networkscited by 82
  9. 2021Advancing Spiking Neural Networks towards Deep Residual Learningcited by 76
  10. 2020DIET-SNN: Direct Input Encoding With Leakage and Threshold Optimization in Deep Spiking Neural Networkscited by 70
  11. 2021HIRE-SNN: Harnessing the Inherent Robustness of Energy-Efficient Deep Spiking Neural Networks by Training with Crafted Input Noisecited by 61
  12. 2022Neuromorphic Data Augmentation for Training Spiking Neural Networkscited by 52
  13. 2019S4NN: temporal backpropagation for spiking neural networks with one spike per neuroncited by 47
  14. 2020Inherent Adversarial Robustness of Deep Spiking Neural Networks: Effects of Discrete Input Encoding and Non-Linear Activationscited by 47
  15. 2019Temporal Coding in Spiking Neural Networks with Alpha Synaptic Function: Learning with Backpropagationcited by 43
  16. 2021Pruning of Deep Spiking Neural Networks through Gradient Rewiringcited by 41
  17. 2016Conversion of Artificial Recurrent Neural Networks to Spiking Neural Networks for Low-power Neuromorphic Hardwarecited by 39
  18. 2022Optimized Potential Initialization for Low-latency Spiking Neural Networkscited by 39
  19. 2020A Tunable Robust Pruning Framework Through Dynamic Network Rewiring of DNNscited by 37
  20. 2021Spiking Neural Networks with Improved Inherent Recurrence Dynamics for Sequential Learningcited by 27
  21. 2021Federated Learning with Spiking Neural Networkscited by 22
  22. 2023Learning to Linearize Deep Neural Networks for Secure and Efficient Private Inferencecited by 20
  23. 2021PrivateSNN: Privacy-Preserving Spiking Neural Networkscited by 15
  24. 2020AttentionLite: Towards Efficient Self-Attention Models for Visioncited by 10
  25. 2021Training Energy-Efficient Deep Spiking Neural Networks with Single-Spike Hybrid Input Encodingcited by 10
  26. 2023To Spike or Not To Spike: A Digital Hardware Perspective on Deep Learning Accelerationcited by 10
  27. 2021Towards Low-Latency Energy-Efficient Deep SNNs via Attention-Guided Compressioncited by 7
  28. 2022Converting Artificial Neural Networks to Spiking Neural Networks via Parameter Calibrationcited by 6
  29. 2022In-Sensor & Neuromorphic Computing are all you need for Energy Efficient Computer Visioncited by 6
  30. 2022IRIS: Integrated Retinal Functionality in Image Sensorscited by 4
  31. 2022Sparse Mixture Once-for-all Adversarial Training for Efficient In-Situ Trade-Off Between Accuracy and Robustness of DNNscited by 4

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 2.

3 papers.