# Conjoint Audio-to-Spikes Encoding and Processing for Efficient Neuromorphic Speech Recognition

arXiv: https://arxiv.org/abs/2608.30792 · HTML: https://mlsanity.com/paper/2608.30792/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 28 of its 58 references that are in the map.
- First posted on arXiv 31 Aug 2026.

## Builds on (most cited first, 28 of 28)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition](https://mlsanity.com/paper/1804.03209/) (2018)
- [Converting Static Image Datasets to Spiking Neuromorphic Datasets Using Saccades](https://mlsanity.com/paper/1507.07629/) (2015)
- [SLAYER: Spike Layer Error Reassignment in Time](https://mlsanity.com/paper/1810.08646/) (2018)
- [Training Spiking Neural Networks Using Lessons From Deep Learning](https://mlsanity.com/paper/2109.12894/) (2021)
- [Video to Events: Recycling Video Datasets for Event Cameras](https://mlsanity.com/paper/1912.03095/) (2019)
- [The Heidelberg spiking datasets for the systematic evaluation of spiking neural networks](https://mlsanity.com/paper/1910.07407/) (2019)
- [A Large Scale Event-based Detection Dataset for Automotive](https://mlsanity.com/paper/2001.08499/) (2020)
- [Learning Delays in Spiking Neural Networks using Dilated Convolutions with Learnable Spacings](https://mlsanity.com/paper/2306.17670/) (2023)
- [Optimizing the energy consumption of spiking neural networks for neuromorphic applications](https://mlsanity.com/paper/1912.01268/) (2019)
- [Scalable Event-by-event Processing of Neuromorphic Sensory Signals With Deep State-Space Models](https://mlsanity.com/paper/2404.18508/) (2024)
- [V2CE: Video to Continuous Events Simulator](https://mlsanity.com/paper/2309.08891/) (2023)
- [Loss shaping enhances exact gradient learning with Eventprop in spiking neural networks](https://mlsanity.com/paper/2212.01232/) (2022)
- [Co-learning synaptic delays, weights and adaptation in spiking neural networks](https://mlsanity.com/paper/2311.16112/) (2023)
- [P-SpikeSSM: Harnessing Probabilistic Spiking State Space Models for Long-Range Dependency Tasks](https://mlsanity.com/paper/2406.02923/) (2024)
- [Advancing Spatio-Temporal Processing in Spiking Neural Networks through Adaptation](https://mlsanity.com/paper/2408.07517/) (2024)
- [Adaptive Axonal Delays in feedforward spiking neural networks for accurate spoken word recognition](https://mlsanity.com/paper/2302.08607/) (2023)
- [An efficient and perceptually motivated auditory neural encoding and decoding algorithm for spiking neural networks](https://mlsanity.com/paper/1909.01302/) (2019)
- [Spiking-LEAF: A Learnable Auditory front-end for Spiking Neural Networks](https://mlsanity.com/paper/2309.09469/) (2023)
- [DelRec: learning delays in recurrent spiking neural networks](https://mlsanity.com/paper/2509.24852/) (2025)
- [Efficient spike encoding algorithms for neuromorphic speech recognition](https://mlsanity.com/paper/2207.07073/) (2022)
- [Low-power SNN-based audio source localisation using a Hilbert Transform spike encoding scheme](https://mlsanity.com/paper/2402.11748/) (2024)
- [SiLIF: Structured State Space Model Dynamics and Parametrization for Spiking Neural Networks](https://mlsanity.com/paper/2506.06374/) (2025)
- [Choice of Mel Filter Bank in Computing MFCC of a Resampled Speech](https://mlsanity.com/paper/1410.6903/) (2014)
- [Efficient Speech Command Recognition Leveraging Spiking Neural Network and Curriculum Learning-based Knowledge Distillation](https://mlsanity.com/paper/2412.12858/) (2024)
- [Evaluation of Neuromorphic Spike Encoding of Sound Using Information Theory](https://mlsanity.com/paper/2202.09619/) (2022)
- [Fundamental Survey on Neuromorphic Based Audio Classification](https://mlsanity.com/paper/2502.15056/) (2025)
- [ASRC-SNN: Adaptive Skip Recurrent Connection Spiking Neural Network](https://mlsanity.com/paper/2505.11455/) (2025)

## Abstract (arXiv)

Obtaining data from neuromorphic sensors and processing it with Spiking Neural Networks is a promising solution to lower the energy cost of artificial intelligence. The current rarity of natively neuromorphic datasets promotes the development of software tools to translate input sensory data into spikes. However, highly bio-mimetic simulators can be challenging to implement on digital hardware. In this work, we evaluate the neuromorphic encoding and subsequent classification of audio into spikes using a non-learnable, high-level, programmable encoder targeting hardware implementation on FPGA. We quantify the pipeline's efficiency with hardware-agnostic metrics based on the quantitative spiking activity. Our study focuses on the simultaneous optimisation of encoder and classifier: the first provides efficient and informative data so that the latter achieves a better performance with an overall lower energy cost at learning and inference. This work introduces the first end-to-end neuromorphic spike-encoding and evaluation of the TIMIT dataset. Our simple feedforward network reaches a classification accuracy of 99.77% on a spike-encoded Heidelberg Digits, overcoming the neuromorphic state of the art on this benchmark dataset.