mlsanityData to 7 Oct 2026

LoAS: Fully Temporal-Parallel Dataflow for Dual-Sparse Spiking Neural Networks

arXiv 2407.14073 ↗First posted 19 Jul 2024cs.AR cs.AI cs.NEcited by 7

Not on the map. Why?

Abstract (arXiv)

Spiking Neural Networks (SNNs) have gained significant research attention in the last decade due to their potential to drive resource-constrained edge devices. Though existing SNN accelerators offer high efficiency in processing sparse spikes with dense weights, opportunities are less explored in SNNs with sparse weights, i.e., dual-sparsity. In this work, we study the acceleration of dual-sparse SNNs, focusing on their core operation, sparse-matrix-sparse-matrix multiplication (spMspM). We observe that naively running a dual-sparse SNN on existing spMspM accelerators designed for dual-sparse Artificial Neural Networks (ANNs) exhibits sub-optimal efficiency. The main challenge is that processing timesteps, a natural property of SNNs, introduces an extra loop to ANN spMspM, leading to longer latency and more memory traffic. To address the problem, we propose a fully temporal-parallel (FTP) dataflow, which minimizes both data movement across timesteps and the end-to-end latency of dual-sparse SNNs. To maximize the efficiency of FTP dataflow, we propose an FTP-friendly spike compression mechanism that efficiently compresses single-bit spikes and ensures contiguous memory access. We further propose an FTP-friendly inner-join circuit that can lower the cost of the expensive prefix-sum circuits with almost no throughput penalty. All the above techniques for FTP dataflow are encapsulated in LoAS, a Low-latency inference Accelerator for dual-sparse SNNs. With FTP dataflow, compression, and inner-join, running dual-sparse SNN workloads on LoAS demonstrates significant speedup (up to 8.51×) and energy reduction (up to 3.68×) compared to running it on prior dual-sparse accelerators.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 32 of its 65 references that are in the map. First posted on arXiv 19 Jul 2024.

Builds on

32 of 65 references.

References · 32

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
  4. 2019Surrogate Gradient Learning in Spiking Neural Networkscited by 604
  5. 2017Spatio-Temporal Backpropagation for Training High-performance Spiking Neural Networkscited by 399
  6. 2021Deep Residual Learning in Spiking Neural Networkscited by 326
  7. 2016EIE: Efficient Inference Engine on Compressed Deep Neural Networkcited by 291
  8. 2020Going Deeper With Directly-Trained Larger Spiking Neural Networkscited by 268
  9. 2018Going Deeper in Spiking Neural Networks: VGG and Residual Architecturescited by 236
  10. 2018Direct Training for Spiking Neural Networks: Faster, Larger, Bettercited by 222
  11. 2022Temporal Efficient Training of Spiking Neural Network via Gradient Re-weightingcited by 192
  12. 2019Spiking-YOLO: Spiking Neural Network for Energy-Efficient Object Detectioncited by 174
20 more
  1. 2023SpikingJelly: An open-source machine learning infrastructure platform for spike-based intelligencecited by 163
  2. 2023Spike-driven Transformercited by 141
  3. 2017SCNN: An Accelerator for Compressed-sparse Convolutional Neural Networkscited by 132
  4. 2020Temporal Spike Sequence Learning via Backpropagation for Deep Spiking Neural Networkscited by 98
  5. 2020Spike-FlowNet: Event-based Optical Flow Estimation with Energy-Efficient Hybrid Neural Networkscited by 98
  6. 2020Revisiting Batch Normalization for Training Low-latency Deep Spiking Neural Networks from Scratchcited by 76
  7. 2021Beyond Classification: Directly Training Spiking Neural Networks for Semantic Segmentationcited by 54
  8. 2020SpArch: Efficient Architecture for Sparse Matrix Multiplicationcited by 47
  9. 2019Temporal Coding in Spiking Neural Networks with Alpha Synaptic Function: Learning with Backpropagationcited by 43
  10. 2021Pruning of Deep Spiking Neural Networks through Gradient Rewiringcited by 41
  11. 2022Exploring Lottery Ticket Hypothesis in Spiking Neural Networkscited by 25
  12. 2022Rate Coding or Direct Coding: Which One is Better for Accurate, Robust, and Energy-efficient Spiking Neural Networks?cited by 24
  13. 2015Stochastic Synapses Enable Efficient Brain-Inspired Learning Machinescited by 21
  14. 2018UCNN: Exploiting Computational Reuse in Deep Neural Networks via Weight Repetitioncited by 19
  15. 2022SATA: Sparsity-Aware Training Accelerator for Spiking Neural Networkscited by 18
  16. 2022Sparseloop: An Analytical Approach To Sparse Tensor Accelerator Modelingcited by 14
  17. 2023Flexagon: A Multi-Dataflow Sparse-Sparse Matrix Multiplication Accelerator for Efficient DNN Processingcited by 13
  18. 2023Workload-Balanced Pruning for Sparse Spiking Neural Networkscited by 13
  19. 2023MINT: Multiplier-less INTeger Quantization for Energy Efficient Spiking Neural Networkscited by 8
  20. 2021Exploiting Activation based Gradient Output Sparsity to Accelerate Backpropagation in CNNscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 1; 2026: 4.

7 papers.

  1. 2024Reconsidering the Energy Efficiency of Spiking Neural Networks Inference from Analytical Perspectivescited by 12
  2. 2024Towards 3D Acceleration for low-power Mixture-of-Experts and Multi-Head Attention Spiking Transformerscited by 1
  3. 2025PASCAL: Precise and Efficient ANN- SNN Conversion using Spike Accumulation and Adaptive Layerwise Activationcited by 1
  4. 2026Surrogates, Spikes, and Sparsity: Performance Analysis and Characterization of SNN Hyperparameters on Hardwarecited by 0
  5. 2026Detecting AI-Generated Videos with Spiking Neural Networkscited by 0
  6. 2026ELSA: An ELastic SNN Inference Architecture for Efficient Neuromorphic Computingcited by 0
  7. 2026NeuroFlex: Lossless Element-Level ANN-SNN Co-Execution for Efficient Sparse Inferencecited by 0