# Towards Unified Neural Decoding of Perceived, Spoken and Imagined Speech from EEG Signals

arXiv: https://arxiv.org/abs/2411.09243 · HTML: https://mlsanity.com/paper/2411.09243/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 22 references that are in the map.
- First posted on arXiv 14 Nov 2024.

## Builds on (most cited first, 7 of 7)

- [EEGNet: A Compact Convolutional Network for EEG-based Brain-Computer Interfaces](https://mlsanity.com/paper/1611.08024/) (2016)
- [Shallow and Deep Convolutional Networks for Saliency Prediction](https://mlsanity.com/paper/1603.00845/) (2016)
- [Towards Voice Reconstruction from EEG during Imagined Speech](https://mlsanity.com/paper/2301.07173/) (2023)
- [Diff-E: Diffusion-based Learning for Decoding Imagined Speech EEG](https://mlsanity.com/paper/2307.14389/) (2023)
- [EEG-ITNet: An Explainable Inception Temporal Convolutional Network for Motor Imagery Classification](https://mlsanity.com/paper/2204.06947/) (2022)
- [SPEAK YOUR MIND! Towards Imagined Speech Recognition With Hierarchical Deep Learning](https://mlsanity.com/paper/1904.05746/) (2019)
- [Neurophysiological Response Based on Auditory Sense for Brain Modulation Using Monaural Beat](https://mlsanity.com/paper/2311.08735/) (2023)

## Built on by (most cited first, 1 of 1)

- [Toward Robust EEG-based Intention Decoding during Misarticulated Speech in Dysarthria](https://mlsanity.com/paper/2511.07895/) (2025)

## Abstract (arXiv)

Brain signals accompany various information relevant to human actions and mental imagery, making them crucial to interpreting and understanding human intentions. Brain-computer interface technology leverages this brain activity to generate external commands for controlling the environment, offering critical advantages to individuals with paralysis or locked-in syndrome. Within the brain-computer interface domain, brain-to-speech research has gained attention, focusing on the direct synthesis of audible speech from brain signals. Most current studies decode speech from brain activity using invasive techniques and emphasize spoken speech data. However, humans express various speech states, and distinguishing these states through non-invasive approaches remains a significant yet challenging task. This research investigated the effectiveness of deep learning models for non-invasive-based neural signal decoding, with an emphasis on distinguishing between different speech paradigms, including perceived, overt, whispered, and imagined speech, across multiple frequency bands. The model utilizing the spatial conventional neural network module demonstrated superior performance compared to other models, especially in the gamma band. Additionally, imagined speech in the theta frequency band, where deep learning also showed strong effects, exhibited statistically significant differences compared to the other speech paradigms.