# Corticomorphic Hybrid CNN-SNN Architecture for EEG-based Low-footprint Low-latency Auditory Attention Detection

arXiv: https://arxiv.org/abs/2307.08501 · HTML: https://mlsanity.com/paper/2307.08501/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 66 references that are in the map.
- First posted on arXiv 13 Jul 2023.

## Builds on (most cited first, 6 of 6)

- [An Analysis of Deep Neural Network Models for Practical Applications](https://mlsanity.com/paper/1605.07678/) (2016)
- [Convolutional Networks for Fast, Energy-Efficient Neuromorphic Computing](https://mlsanity.com/paper/1603.08270/) (2016)
- [Training Spiking Deep Networks for Neuromorphic Hardware](https://mlsanity.com/paper/1611.05141/) (2016)
- [NengoDL: Combining deep learning and neuromorphic modelling methods](https://mlsanity.com/paper/1805.11144/) (2018)
- [Auditory Attention Decoding from EEG using Convolutional Recurrent Neural Network](https://mlsanity.com/paper/2103.02183/) (2021)
- [EEG-assisted Modulation of Sound Sources in the Auditory Scene](https://mlsanity.com/paper/1612.00703/) (2016)

## Built on by (most cited first, 2 of 2)

- [Spiking-PhysFormer: Camera-Based Remote Photoplethysmography with Parallel Spike-driven Transformer](https://mlsanity.com/paper/2402.04798/) (2024)
- [FAConformer: Frequency-Aware Convolutional Transformer for Auditory Attention Decoding](https://mlsanity.com/paper/2606.14120/) (2026)

## Abstract (arXiv)

In a multi-speaker "cocktail party" scenario, a listener can selectively attend to a speaker of interest. Studies into the human auditory attention network demonstrate cortical entrainment to speech envelopes resulting in highly correlated Electroencephalography (EEG) measurements. Current trends in EEG-based auditory attention detection (AAD) using artificial neural networks (ANN) are not practical for edge-computing platforms due to longer decision windows using several EEG channels, with higher power consumption and larger memory footprint requirements. Nor are ANNs capable of accurately modeling the brain's top-down attention network since the cortical organization is complex and layer. In this paper, we propose a hybrid convolutional neural network-spiking neural network (CNN-SNN) corticomorphic architecture, inspired by the auditory cortex, which uses EEG data along with multi-speaker speech envelopes to successfully decode auditory attention with low latency down to 1 second, using only 8 EEG electrodes strategically placed close to the auditory cortex, at a significantly higher accuracy of 91.03%, compared to the state-of-the-art. Simultaneously, when compared to a traditional CNN reference model, our model uses ~15% fewer parameters at a lower bit precision resulting in ~57% memory footprint reduction. The results show great promise for edge-computing in brain-embedded devices, like smart hearing aids.