# SpikeNAS: A Fast Memory-Aware Neural Architecture Search Framework for Spiking Neural Network-based Embedded AI Systems

arXiv: https://arxiv.org/abs/2402.11322 · HTML: https://mlsanity.com/paper/2402.11322/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 56 references that are in the map.
- First posted on arXiv 17 Feb 2024.

## Builds on (most cited first, 30 of 33)

- [Surrogate Gradient Learning in Spiking Neural Networks](https://mlsanity.com/paper/1901.09948/) (2019)
- [NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Search](https://mlsanity.com/paper/2001.00326/) (2020)
- [Incorporating Learnable Membrane Time Constant to Enhance Learning of Spiking Neural Networks](https://mlsanity.com/paper/2007.05785/) (2020)
- [Going Deeper With Directly-Trained Larger Spiking Neural Networks](https://mlsanity.com/paper/2011.05280/) (2020)
- [Deep Learning in Spiking Neural Networks](https://mlsanity.com/paper/1804.08150/) (2018)
- [Neural Architecture Search without Training](https://mlsanity.com/paper/2006.04647/) (2020)
- [Direct Training for Spiking Neural Networks: Faster, Larger, Better](https://mlsanity.com/paper/1809.05793/) (2018)
- [SpikingJelly: An open-source machine learning infrastructure platform for spike-based intelligence](https://mlsanity.com/paper/2310.16620/) (2023)
- [Enabling Spike-based Backpropagation for Training Deep Neural Network Architectures](https://mlsanity.com/paper/1903.06379/) (2019)
- [Object Detection with Spiking Neural Networks on Automotive Event Data](https://mlsanity.com/paper/2205.04339/) (2022)
- [Neural Architecture Search for Spiking Neural Networks](https://mlsanity.com/paper/2201.10355/) (2022)
- [AutoSNN: Towards Energy-Efficient Spiking Neural Networks](https://mlsanity.com/paper/2201.12738/) (2022)
- [Q-SpiNN: A Framework for Quantizing Spiking Neural Networks](https://mlsanity.com/paper/2107.01807/) (2021)
- [FSpiNN: An Optimization Framework for Memory- and Energy-Efficient Spiking Neural Networks](https://mlsanity.com/paper/2007.08860/) (2020)
- [SpikeDyn: A Framework for Energy-Efficient Spiking Neural Networks with Continual and Unsupervised Learning Capabilities in Dynamic Environments](https://mlsanity.com/paper/2103.00424/) (2021)
- [Spiking Neural Network Integrated Circuits: A Review of Trends and Future Directions](https://mlsanity.com/paper/2203.07006/) (2022)
- [Darwin3: A large-scale neuromorphic chip with a Novel ISA and On-Chip Learning](https://mlsanity.com/paper/2312.17582/) (2023)
- [SparkXD: A Framework for Resilient and Energy-Efficient Spiking Neural Network Inference using Approximate DRAM](https://mlsanity.com/paper/2103.00421/) (2021)
- [ReSpawn: Energy-Efficient Fault-Tolerance for Spiking Neural Networks considering Unreliable Memories](https://mlsanity.com/paper/2108.10271/) (2021)
- [SATA: Sparsity-Aware Training Accelerator for Spiking Neural Networks](https://mlsanity.com/paper/2204.05422/) (2022)
- [SoftSNN: Low-Cost Fault Tolerance for Spiking Neural Network Accelerators under Soft Errors](https://mlsanity.com/paper/2203.05523/) (2022)
- [lpSpikeCon: Enabling Low-Precision Spiking Neural Network Processing for Efficient Unsupervised Continual Learning on Autonomous Agents](https://mlsanity.com/paper/2205.12295/) (2022)
- [Towards Energy-Efficient and Secure Edge AI: A Cross-Layer Framework](https://mlsanity.com/paper/2109.09829/) (2021)
- [Workload-Balanced Pruning for Sparse Spiking Neural Networks](https://mlsanity.com/paper/2302.06746/) (2023)
- [TopSpark: A Timestep Optimization Methodology for Energy-Efficient Spiking Neural Networks on Autonomous Mobile Agents](https://mlsanity.com/paper/2303.01826/) (2023)
- [MINT: Multiplier-less INTeger Quantization for Energy Efficient Spiking Neural Networks](https://mlsanity.com/paper/2305.09850/) (2023)
- [SpikeSim: An end-to-end Compute-in-Memory Hardware Evaluation Tool for Benchmarking Spiking Neural Networks](https://mlsanity.com/paper/2210.12899/) (2022)
- [EnforceSNN: Enabling Resilient and Energy-Efficient Spiking Neural Network Inference considering Approximate DRAMs for Embedded Systems](https://mlsanity.com/paper/2304.04039/) (2023)
- [RescueSNN: Enabling Reliable Executions on Spiking Neural Network Accelerators under Permanent Faults](https://mlsanity.com/paper/2304.04041/) (2023)
- [AutoST: Training-free Neural Architecture Search for Spiking Transformers](https://mlsanity.com/paper/2307.00293/) (2023)

## Built on by (most cited first, 12 of 12)

- [Continual Learning with Neuromorphic Computing: Foundations, Methods, and Emerging Applications](https://mlsanity.com/paper/2410.09218/) (2024)
- [QSLM: A Performance- and Memory-aware Quantization Framework with Tiered Search Strategy for Spike-driven Language Models](https://mlsanity.com/paper/2601.00679/) (2026)
- [Enabling Energy-Efficient Simultaneous Multi-Task Reinforcement Learning through Spiking Neural Networks with Active Dendrites for Bio-inspired Generalist Agents](https://mlsanity.com/paper/2412.04847/) (2024)
- [LightSNN: Lightweight Architecture Search for Sparse and Accurate Spiking Neural Networks](https://mlsanity.com/paper/2503.21846/) (2025)
- [Scalable Multi-Task Learning through Spiking Neural Networks with Adaptive Task-Switching Policy for Intelligent Autonomous Agents](https://mlsanity.com/paper/2504.13541/) (2025)
- [SpikeVox: Towards Energy-Efficient Speech Therapy Framework with Spike-driven Generative Language Models](https://mlsanity.com/paper/2510.15566/) (2025)
- [Focus Session: Hardware and Software Techniques for Accelerating Multimodal Foundation Models](https://mlsanity.com/paper/2604.21952/) (2026)
- [PSViT: A Methodology for Structurally Pruning Spiking Vision Transformers](https://mlsanity.com/paper/2606.03257/) (2026)
- [PrimeSVT: An Automated Memory-aware Pruning Framework with Prioritized Compression Policy for Spiking Vision Transformers](https://mlsanity.com/paper/2606.03428/) (2026)
- [AQ4SViT: An Automated Quantization Framework with Search Gating Policy for Compressing Spiking Vision Transformers](https://mlsanity.com/paper/2606.15523/) (2026)
- [MorphAtt: A Neuromorphic Accelerator for Efficient Multi-Head Attention Processing in Spiking Vision Transformers](https://mlsanity.com/paper/2609.33207/) (2026)
- [MTLiquid: Enabling Efficient Multi-Task Learning using Liquid Neural Networks for Lightweight Healthcare Monitoring Systems](https://mlsanity.com/paper/2609.33232/) (2026)

## Abstract (arXiv)

Embedded AI systems are expected to incur low power/energy consumption for solving machine learning tasks, as these systems are usually power constrained (e.g., object recognition task in autonomous mobile agents with portable batteries). These requirements can be fulfilled by Spiking Neural Networks (SNNs), since their bio-inspired spike-based operations offer high accuracy and ultra low-power/energy computation. Currently, most of SNN architectures are derived from Artificial Neural Networks whose neurons' architectures and operations are different from SNNs, and/or developed without considering memory budgets from the underlying processing hardware of embedded platforms. These limitations hinder SNNs from reaching their full potential in accuracy and efficiency. Toward this, we propose SpikeNAS, a novel fast memory-aware neural architecture search (NAS) framework for SNNs that quickly finds an appropriate SNN architecture with high accuracy under the given memory budgets from targeted embedded systems. To do this, our SpikeNAS employs several key steps: analyzing the impacts of network operations on the accuracy, enhancing the network architecture to improve the learning quality, developing a fast memory-aware search algorithm, and performing quantization. The experimental results show that our SpikeNAS improves the searching time and maintains high accuracy compared to state-of-the-art while meeting the given memory budgets (e.g., 29x, 117x, and 3.7x faster search for CIFAR10, CIFAR100, and TinyImageNet200 respectively, using an Nvidia RTX A6000 GPU machine), thereby quickly providing the appropriate SNN architecture for the memory-constrained embedded AI systems.