mlsanityData to 14 Oct 2026

Neural Architecture Search for Spiking Neural Networks

arXiv 2201.10355 ↗First posted 23 Jan 2022cs.NE cs.AI cs.LGcited by 53

Not on the map. Why?

Abstract (arXiv)

Spiking Neural Networks (SNNs) have gained huge attention as a potential energy-efficient alternative to conventional Artificial Neural Networks (ANNs) due to their inherent high-sparsity activation. However, most prior SNN methods use ANN-like architectures (e.g., VGG-Net or ResNet), which could provide sub-optimal performance for temporal sequence processing of binary information in SNNs. To address this, in this paper, we introduce a novel Neural Architecture Search (NAS) approach for finding better SNN architectures. Inspired by recent NAS approaches that find the optimal architecture from activation patterns at initialization, we select the architecture that can represent diverse spike activation patterns across different data samples without training. Moreover, to further leverage the temporal information among the spikes, we search for feed forward connections as well as backward connections (i.e., temporal feedback connections) between layers. Interestingly, SNASNet found by our search algorithm achieves higher performance with backward connections, demonstrating the importance of designing SNN architecture for suitably using temporal information. We conduct extensive experiments on three image recognition benchmarks where we show that SNASNet achieves state-of-the-art performance with significantly lower timesteps (5 timesteps). Code is available at Github.

Key facts. Cited by 53 papers in the map (data through 14 Oct 2026). Builds on 73 of its 114 references that are in the map. First posted on arXiv 23 Jan 2022.

Builds on

73 of 114 references.

References · 73

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  4. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  5. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,611
  6. 2018DARTS: Differentiable Architecture Searchcited by 1,798
  7. 2016Neural Architecture Search with Reinforcement Learningcited by 1,499
  8. 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,101
  9. 2018MnasNet: Platform-Aware Neural Architecture Search for Mobilecited by 943
  10. 2018Regularized Evolution for Image Classifier Architecture Searchcited by 909
  11. 2018Efficient Neural Architecture Search via Parameter Sharingcited by 837
  12. 2018ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardwarecited by 673
38 more
  1. 2019Surrogate Gradient Learning in Spiking Neural Networkscited by 626
  2. 2018FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Searchcited by 485
  3. 2020NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Searchcited by 416
  4. 2019Single Path One-Shot Neural Architecture Search with Uniform Samplingcited by 414
  5. 2017Spatio-Temporal Backpropagation for Training High-performance Spiking Neural Networkscited by 406
  6. 2014On the Number of Linear Regions of Deep Neural Networkscited by 381
  7. 2020Incorporating Learnable Membrane Time Constant to Enhance Learning of Spiking Neural Networkscited by 345
  8. 2019NAS-Bench-101: Towards Reproducible Neural Architecture Searchcited by 344
  9. 2021Deep Residual Learning in Spiking Neural Networkscited by 331
  10. 2016On the Expressive Power of Deep Neural Networkscited by 306
  11. 2018SNAS: Stochastic Neural Architecture Searchcited by 289
  12. 2016Designing Neural Network Architectures using Reinforcement Learningcited by 279
  13. 2020Going Deeper With Directly-Trained Larger Spiking Neural Networkscited by 278
  14. 2018SLAYER: Spike Layer Error Reassignment in Timecited by 266
  15. 2019Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentationcited by 260
  16. 2018Going Deeper in Spiking Neural Networks: VGG and Residual Architecturescited by 245
  17. 2020Neural Architecture Search without Trainingcited by 244
  18. 2018Direct Training for Spiking Neural Networks: Faster, Larger, Bettercited by 233
  19. 2016Training Deep Spiking Neural Networks using Backpropagationcited by 205
  20. 2017SMASH: One-Shot Model Architecture Search through HyperNetworkscited by 202
  21. 2021Zero-Cost Proxies for Lightweight NAScited by 199
  22. 2021Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspectivecited by 169
  23. 2019Complexity of Linear Regions in Deep Networkscited by 167
  24. 2019Deep ReLU Networks Have Surprisingly Few Activation Patternscited by 161
  25. 2020RMP-SNN: Residual Membrane Potential Neuron for Enabling Deeper High-Accuracy and Low-Latency Spiking Neural Networkcited by 145
  26. 2020Enabling Deep Spiking Neural Networks with Hybrid Conversion and Spike Timing Dependent Backpropagationcited by 144
  27. 2021Optimal Conversion of Conventional Artificial Neural Networks to Spiking Neural Networkscited by 139
  28. 2019Enabling Spike-based Backpropagation for Training Deep Neural Network Architecturescited by 123
  29. 2021A Free Lunch From ANN: Towards Efficient, Accurate Spiking Neural Networks Calibrationcited by 117
  30. 2021Temporal-wise Attention Spiking Neural Networks for Event Streams Classificationcited by 106
  31. 2020Temporal Spike Sequence Learning via Backpropagation for Deep Spiking Neural Networkscited by 100
  32. 2019AutoGAN: Neural Architecture Search for Generative Adversarial Networkscited by 94
  33. 2019DetNAS: Backbone Search for Object Detectioncited by 81
  34. 2020Revisiting Batch Normalization for Training Low-latency Deep Spiking Neural Networks from Scratchcited by 80
  35. 2016Supervised learning based on temporal coding in spiking neural networkscited by 76
  36. 2020DSNAS: Direct Neural Architecture Search without Parameter Retrainingcited by 69
  37. 2021HIRE-SNN: Harnessing the Inherent Robustness of Energy-Efficient Deep Spiking Neural Networks by Training with Crafted Input Noisecited by 65
  38. 2021TransNAS-Bench-101: Improving Transferability and Generalizability of Cross-Task Neural Architecture Searchcited by 58

50 most cited of 73 All 73 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 10; 2023: 22; 2024: 12; 2025: 7; 2026: 2.

53 papers.

Sort:

By area:

  1. 2022Temporal Efficient Training of Spiking Neural Network via Gradient Re-weightingcited by 198
  2. 2023SpikingJelly: An open-source machine learning infrastructure platform for spike-based intelligencecited by 169
  3. 2022Online Training Through Time for Spiking Neural Networkscited by 49
  4. 2023Fast-SNN: Fast Spiking Neural Network by Converting Quantized ANNcited by 46
  5. 2023SEENN: Towards Temporal Spiking Early-Exit Neural Networkscited by 41
  6. 2022AutoSNN: Towards Energy-Efficient Spiking Neural Networkscited by 40
  7. 2023Direct Learning-Based Deep Spiking Neural Networks: A Reviewcited by 31
  8. 2022Exploring Lottery Ticket Hypothesis in Spiking Neural Networkscited by 26
  9. 2022Exploring Temporal Information Dynamics in Spiking Neural Networkscited by 24
  10. 2023SpikeBERT: A Language Spikformer Learned from BERT with Knowledge Distillationcited by 17
  11. 2024EAS-SNN: End-to-End Adaptive Sampling and Representation for Event-based Detection with Recurrent Spiking Neural Networkscited by 16
  12. 2023Unleashing the Potential of Spiking Neural Networks by Dynamic Confidencecited by 15
38 more
  1. 2023Inherent Redundancy in Spiking Neural Networkscited by 13
  2. 2023A Unified Framework for Soft Threshold Pruningcited by 12
  3. 2024SpikeNAS: A Fast Memory-Aware Neural Architecture Search Framework for Spiking Neural Network-based Embedded AI Systemscited by 12
  4. 2023Uncovering the Representation of Spiking Neural Networks Trained with Surrogate Gradientcited by 11
  5. 2022Meta-learning Spiking Neural Networks with Surrogate Gradient Descentcited by 10
  6. 2023Temporal Knowledge Sharing enable Spiking Neural Network Learning from Past and Futurecited by 10
  7. 2023SPIDE: A Purely Spike-based Method for Training Feedback Spiking Neural Networkscited by 8
  8. 2023Enabling Resource-efficient AIoT System with Cross-level Optimization: A surveycited by 8
  9. 2023Tensor Decomposition Based Attention Module for Spiking Neural Networkscited by 8
  10. 2022Converting Artificial Neural Networks to Spiking Neural Networks via Parameter Calibrationcited by 7
  11. 2023Efficient Deep Spiking Multi-Layer Perceptrons with Multiplication-Free Inferencecited by 7
  12. 2024Embodied Neuromorphic Artificial Intelligence for Robotics: Perspectives, Challenges, and Research Development Stackcited by 7
  13. 2024Toward Large-scale Spiking Neural Networks: A Comprehensive Survey and Future Directionscited by 7
  14. 2022Wearable-based Human Activity Recognition with Spatio-Temporal Spiking Neural Networkscited by 6
  15. 2023AutoST: Training-free Neural Architecture Search for Spiking Transformerscited by 6
  16. 2023Memory-Efficient Reversible Spiking Neural Networkscited by 6
  17. 2022Hoyer regularizer is all you need for ultra low-latency spiking neural networkscited by 4
  18. 2025Time to Spike? Understanding the Representational Power of Spiking Neural Networks in Discrete Timecited by 4
  19. 2023Exploiting High Performance Spiking Neural Networks with Efficient Spiking Patternscited by 3
  20. 2023Multi-scale Evolutionary Neural Architecture Search for Deep Spiking Neural Networkscited by 3
  21. 2024SpikeExplorer: hardware-oriented Design Space Exploration for Spiking Neural Networks on FPGAcited by 3
  22. 2025MoCom: Motion-based Inter-MAV Visual Communication Using Event Vision and Spiking Neural Networkscited by 3
  23. 2024SpiKernel: A Kernel Size Exploration Methodology for Improving Accuracy of the Embedded Spiking Neural Network Systemscited by 2
  24. 2024Direct Training Needs Regularisation: Anytime Optimal Inference Spiking Neural Networkcited by 2
  25. 2024FastSpiker: Enabling Fast Training for Spiking Neural Networks on Event-based Data through Learning Rate Enhancements for Autonomous Embedded Systemscited by 2
  26. 2025Edge Intelligence with Spiking Neural Networkscited by 2
  27. 2022S3NN: Time Step Reduction of Spiking Surrogate Gradients for Training Energy Efficient Single-Step Spiking Neural Networkscited by 1
  28. 2023Skip Connections in Spiking Neural Networks: An Analysis of Their Effect on Network Trainingcited by 1
  29. 2023Brain-inspired Evolutionary Architectures for Spiking Neural Networkscited by 1
  30. 2024Parallel Hyperparameter Optimization Of Spiking Neural Networkcited by 1
  31. 2024NeuroNAS: Enhancing Efficiency of Neuromorphic In-Memory Computing for Intelligent Mobile Agents through Hardware-Aware Spiking Neural Architecture Searchcited by 1
  32. 2024When In-memory Computing Meets Spiking Neural Networks -- A Perspective on Device-Circuit-System-and-Algorithm Co-designcited by 1
  33. 2025LightSNN: Lightweight Architecture Search for Sparse and Accurate Spiking Neural Networkscited by 1
  34. 2025Meta knowledge assisted Evolutionary Neural Architecture Searchcited by 1
  35. 2026Energy-Efficient Neuromorphic Computing for Edge AI: A Framework with Adaptive Spiking Neural Networks and Hardware-Aware Optimizationcited by 1
  36. 2023SPENSER: Towards a NeuroEvolutionary Approach for Convolutional Spiking Neural Networkscited by 0
  37. 2023Autonomous Driving using Spiking Neural Networks on Dynamic Vision Sensor Data: A Case Study of Traffic Light Change Detectioncited by 0
  38. 2024Differentiable architecture search with multi-dimensional attention for spiking neural networkscited by 0

50 most cited of 53 All 53 (JSON)