# End-to-End Keyword Spotting on FPGA Using Graph Neural Networks with a Neuromorphic Auditory Sensor

arXiv: https://arxiv.org/abs/2605.09570 · HTML: https://mlsanity.com/paper/2605.09570/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 35 references that are in the map.
- First posted on arXiv 10 May 2026.

## Builds on (most cited first, 17 of 17)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition](https://mlsanity.com/paper/1804.03209/) (2018)
- [The Heidelberg spiking datasets for the systematic evaluation of spiking neural networks](https://mlsanity.com/paper/1910.07407/) (2019)
- [Accurate and efficient time-domain classification with adaptive spiking recurrent neural networks](https://mlsanity.com/paper/2103.12593/) (2021)
- [Learning Delays in Spiking Neural Networks using Dilated Convolutions with Learnable Spacings](https://mlsanity.com/paper/2306.17670/) (2023)
- [Scalable Event-by-event Processing of Neuromorphic Sensory Signals With Deep State-Space Models](https://mlsanity.com/paper/2404.18508/) (2024)
- [Spiker+: a framework for the generation of efficient Spiking Neural Networks FPGA accelerators for inference at the edge](https://mlsanity.com/paper/2401.01141/) (2024)
- [Fluctuation-driven initialization for spiking neural network training](https://mlsanity.com/paper/2206.10226/) (2022)
- [Advancing Spatio-Temporal Processing in Spiking Neural Networks through Adaptation](https://mlsanity.com/paper/2408.07517/) (2024)
- [A Fully-Configurable Open-Source Software-Defined Digital Quantized Spiking Neural Core Architecture](https://mlsanity.com/paper/2404.02248/) (2024)
- [TSkips: Efficiency Through Explicit Temporal Delay Connections in Spiking Neural Networks](https://mlsanity.com/paper/2411.16711/) (2024)
- [Scaling Up Resonate-and-Fire Networks for Fast Deep Learning](https://mlsanity.com/paper/2504.00719/) (2025)
- [FPGA Implementation of the CAR Model of the Cochlea](https://mlsanity.com/paper/1503.00504/) (2015)
- [Hardware-accelerated graph neural networks: an alternative approach for neuromorphic event-based audio classification and keyword spotting on SoC FPGA](https://mlsanity.com/paper/2602.16442/) (2026)
- [Hardware-Accelerated Event-Graph Neural Networks for Low-Latency Time-Series Classification on SoC FPGA](https://mlsanity.com/paper/2503.06629/) (2025)

## Abstract (arXiv)

With the rapid growth of mobile robotics and embedded intelligence, there is an increasing demand for efficient on-device data processing on edge platforms. A promising research direction is the use of neuromorphic sensors inspired by human sensory systems, which generate sparse, event-based data encoding changes in the environment. In this work, we present the first end-to-end FPGA implementation of a keyword spotting system that integrates a Neuromorphic Auditory Sensor (NAS) and a graph neural network (GNN) on a single FPGA device, enabling real-time processing of raw audio data. The proposed architecture eliminates conventional signal preprocessing and operates directly on event-based audio streams. Leveraging a compute-near-memory network architecture, the system achieves efficient inference with low latency and low power consumption. Experimental results demonstrate an accuracy of 87.43% after quantization on the Google Speech Commands v2 dataset processed through the neuromorphic sensor, with end-to-end latency below 35 us and average power consumption of 1.12 W. The processed datasets, software models, and hardware modules are available at https://github.com/vision-agh/NAS-GNN-KWS.