# Decoding Covert Speech from EEG Using a Functional Areas Spatio-Temporal Transformer

arXiv: https://arxiv.org/abs/2504.03762 · HTML: https://mlsanity.com/paper/2504.03762/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 65 references that are in the map.
- First posted on arXiv 2 Apr 2025.

## Builds on (most cited first, 14 of 14)

- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Image Transformer](https://mlsanity.com/paper/1802.05751/) (2018)
- [EEGNet: A Compact Convolutional Network for EEG-based Brain-Computer Interfaces](https://mlsanity.com/paper/1611.08024/) (2016)
- [EEG-Based Emotion Recognition Using Regularized Graph Neural Networks](https://mlsanity.com/paper/1907.07835/) (2019)
- [Transformer-based Spatial-Temporal Feature Learning for EEG Decoding](https://mlsanity.com/paper/2106.11170/) (2021)
- [TSception: Capturing Temporal Dynamics and Spatial Asymmetry from EEG for Emotion Recognition](https://mlsanity.com/paper/2104.02935/) (2021)
- [LGGNet: Learning from Local-Global-Graph Representations for Brain-Computer Interface](https://mlsanity.com/paper/2105.02786/) (2021)
- [EEG-Deformer: A Dense Convolutional Transformer for Brain-computer Interfaces](https://mlsanity.com/paper/2405.00719/) (2024)
- [EmT: A Novel Transformer for Generalized Cross-subject EEG Emotion Recognition](https://mlsanity.com/paper/2406.18345/) (2024)
- [EEG-based Brain-Computer Interfaces (BCIs): A Survey of Recent Studies on Signal Sensing Technologies and Computational Intelligence Approaches and their Applications](https://mlsanity.com/paper/2001.11337/) (2020)
- [Diff-E: Diffusion-based Learning for Decoding Imagined Speech EEG](https://mlsanity.com/paper/2307.14389/) (2023)
- [MASA-TCN: Multi-anchor Space-aware Temporal Convolutional Neural Networks for Continuous and Discrete EEG Emotion Recognition](https://mlsanity.com/paper/2308.16207/) (2023)
- [ViT2EEG: Leveraging Hybrid Pretrained Vision Transformers for EEG Data](https://mlsanity.com/paper/2308.00454/) (2023)
- [Towards Neural Decoding of Imagined Speech based on Spoken Speech](https://mlsanity.com/paper/2212.02047/) (2022)

## Built on by (most cited first, 2 of 2)

- [ECHO: Toward Contextual Seq2Seq Paradigms in Large EEG Models](https://mlsanity.com/paper/2509.22556/) (2025)
- [Toward Practical BCI: A Real-time Wireless Imagined Speech EEG Decoding System](https://mlsanity.com/paper/2511.07936/) (2025)

## Abstract (arXiv)

Covert speech involves imagining speaking without audible sound or any movements. Decoding covert speech from electroencephalogram (EEG) is challenging due to a limited understanding of neural pronunciation mapping and the low signal-to-noise ratio of the signal. In this study, we developed a large-scale multi-utterance speech EEG dataset from 57 right-handed native English-speaking subjects, each performing covert and overt speech tasks by repeating the same word in five utterances within a ten-second duration. Given the spatio-temporal nature of the neural activation process during speech pronunciation, we developed a Functional Areas Spatio-temporal Transformer (FAST), an effective framework for converting EEG signals into tokens and utilizing transformer architecture for sequence encoding. Our results reveal distinct and interpretable speech neural features by the visualization of FAST-generated activation maps across frontal and temporal brain regions with each word being covertly spoken, providing new insights into the discriminative features of the neural representation of covert speech. This is the first report of such a study, which provides interpretable evidence for speech decoding from EEG. The code for this work has been made public at https://github.com/Jiang-Muyun/FAST