# EEG-Transformer: Self-attention from Transformer Architecture for Decoding EEG of Imagined Speech

arXiv: https://arxiv.org/abs/2112.09239 · HTML: https://mlsanity.com/paper/2112.09239/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Dec 2021.

## Built on by (most cited first, 14 of 14)

- [EEG-Deformer: A Dense Convolutional Transformer for Brain-computer Interfaces](https://mlsanity.com/paper/2405.00719/) (2024)
- [Diff-E: Diffusion-based Learning for Decoding Imagined Speech EEG](https://mlsanity.com/paper/2307.14389/) (2023)
- [BrainRVQ: A High-Fidelity EEG Foundation Model via Dual-Domain Residual Quantization and Hierarchical Autoregression](https://mlsanity.com/paper/2602.16951/) (2026)
- [Transformer-based EEG Decoding: A Survey](https://mlsanity.com/paper/2507.02320/) (2025)
- [Deep learning for ECoG brain-computer interface: end-to-end vs. hand-crafted features](https://mlsanity.com/paper/2210.02544/) (2022)
- [EVA-Net: Interpretable Anomaly Detection for Brain Health via Learning Continuous Aging Prototypes from One-Class EEG Cohorts](https://mlsanity.com/paper/2511.15393/) (2025)
- [MultiDiffNet: A Multi-Objective Diffusion Framework for Generalizable Brain Decoding](https://mlsanity.com/paper/2511.18294/) (2025)
- [BrainStack: Neuro-MoE with Functionally Guided Expert Routing for EEG-Based Language Decoding](https://mlsanity.com/paper/2601.21148/) (2026)
- [An Interpretable and Attention-based Method for Gaze Estimation Using Electroencephalography](https://mlsanity.com/paper/2308.05768/) (2023)
- [Neural Speech Embeddings for Speech Synthesis Based on Deep Generative Networks](https://mlsanity.com/paper/2312.05814/) (2023)
- [Brain-inspired Computing Based on Deep Learning for Human-computer Interaction: A Review](https://mlsanity.com/paper/2312.07213/) (2023)
- [Biaxialformer: Leveraging Channel Independence and Inter-Channel Correlations in EEG Signal Decoding for Predicting Neurological Outcomes](https://mlsanity.com/paper/2507.02879/) (2025)
- [EEG-Based Brain-LLM Interface for Human Preference Aligned Generation](https://mlsanity.com/paper/2603.16897/) (2026)
- [Transformer Based Model for Spatiotemporal Feature Learning in EEG Emotion Recognition](https://mlsanity.com/paper/2606.10718/) (2026)

## Abstract (arXiv)

Transformers are groundbreaking architectures that have changed a flow of deep learning, and many high-performance models are developing based on transformer architectures. Transformers implemented only with attention with encoder-decoder structure following seq2seq without using RNN, but had better performance than RNN. Herein, we investigate the decoding technique for electroencephalography (EEG) composed of self-attention module from transformer architecture during imagined speech and overt speech. We performed classification of nine subjects using convolutional neural network based on EEGNet that captures temporal-spectral-spatial features from EEG of imagined speech and overt speech. Furthermore, we applied the self-attention module to decoding EEG to improve the performance and lower the number of parameters. Our results demonstrate the possibility of decoding brain activities of imagined speech and overt speech using attention modules. Also, only single channel EEG or ear-EEG can be used to decode the imagined speech for practical BCIs.