# SPEAK YOUR MIND! Towards Imagined Speech Recognition With Hierarchical Deep Learning

arXiv: https://arxiv.org/abs/1904.05746 · HTML: https://mlsanity.com/paper/1904.05746/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Apr 2019.

## Built on by (most cited first, 5 of 5)

- [SEE: Semantically Aligned EEG-to-Text Translation](https://mlsanity.com/paper/2409.16312/) (2024)
- [Inner speech recognition through electroencephalographic signals](https://mlsanity.com/paper/2210.06472/) (2022)
- [Towards Unified Neural Decoding of Perceived, Spoken and Imagined Speech from EEG Signals](https://mlsanity.com/paper/2411.09243/) (2024)
- [Reconstructing Unseen Sentences from Speech-related Biosignals for Open-vocabulary Neural Communication](https://mlsanity.com/paper/2510.27247/) (2025)
- [Distinct Theta Synchrony across Speech Modes: Perceived, Spoken, Whispered, and Imagined](https://mlsanity.com/paper/2511.07918/) (2025)

## Abstract (arXiv)

Speech-related Brain Computer Interface (BCI) technologies provide effective vocal communication strategies for controlling devices through speech commands interpreted from brain signals. In order to infer imagined speech from active thoughts, we propose a novel hierarchical deep learning BCI system for subject-independent classification of 11 speech tokens including phonemes and words. Our novel approach exploits predicted articulatory information of six phonological categories (e.g., nasal, bilabial) as an intermediate step for classifying the phonemes and words, thereby finding discriminative signal responsible for natural speech synthesis. The proposed network is composed of hierarchical combination of spatial and temporal CNN cascaded with a deep autoencoder. Our best models on the KARA database achieve an average accuracy of 83.42% across the six different binary phonological classification tasks, and 53.36% for the individual token identification task, significantly outperforming our baselines. Ultimately, our work suggests the possible existence of a brain imagery footprint for the underlying articulatory movement related to different sounds that can be used to aid imagined speech decoding.