# EEG-Deformer: A Dense Convolutional Transformer for Brain-computer Interfaces

arXiv: https://arxiv.org/abs/2405.00719 · HTML: https://mlsanity.com/paper/2405.00719/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 43 references that are in the map.
- First posted on arXiv 25 Apr 2024.

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [EEGNet: A Compact Convolutional Network for EEG-based Brain-Computer Interfaces](https://mlsanity.com/paper/1611.08024/) (2016)
- [TSception: Capturing Temporal Dynamics and Spatial Asymmetry from EEG for Emotion Recognition](https://mlsanity.com/paper/2104.02935/) (2021)
- [LGGNet: Learning from Local-Global-Graph Representations for Brain-Computer Interface](https://mlsanity.com/paper/2105.02786/) (2021)
- [Multi-channel EEG recordings during a sustained-attention driving task](https://mlsanity.com/paper/1809.06534/) (2018)
- [MIN2Net: End-to-End Multi-Task Learning for Subject-Independent Motor Imagery EEG Classification](https://mlsanity.com/paper/2102.03814/) (2021)
- [HetEmotionNet: Two-Stream Heterogeneous Graph Recurrent Neural Network for Multi-modal Emotion Recognition](https://mlsanity.com/paper/2108.03354/) (2021)
- [EEG-Transformer: Self-attention from Transformer Architecture for Decoding EEG of Imagined Speech](https://mlsanity.com/paper/2112.09239/) (2021)
- [A Transformer-based deep neural network model for SSVEP classification](https://mlsanity.com/paper/2210.04172/) (2022)
- [AMDET: Attention based Multiple Dimensions EEG Transformer for Emotion Recognition](https://mlsanity.com/paper/2212.12134/) (2022)
- [MASA-TCN: Multi-anchor Space-aware Temporal Convolutional Neural Networks for Continuous and Discrete EEG Emotion Recognition](https://mlsanity.com/paper/2308.16207/) (2023)
- [Emotion self-regulation training in major depressive disorder using simultaneous real-time fMRI and EEG neurofeedback](https://mlsanity.com/paper/1909.05764/) (2019)

## Built on by (most cited first, 26 of 26)

- [CodeBrain: Bridging Decoupled Tokenizer and Multi-Scale Architecture for EEG Foundation Model](https://mlsanity.com/paper/2506.09110/) (2025)
- [DBConformer: Dual-Branch Convolutional Transformer for EEG Decoding](https://mlsanity.com/paper/2506.21140/) (2025)
- [ECHO: Toward Contextual Seq2Seq Paradigms in Large EEG Models](https://mlsanity.com/paper/2509.22556/) (2025)
- [Neural Spelling: A Spell-Based BCI System for Language Neural Decoding](https://mlsanity.com/paper/2501.17489/) (2025)
- [Transformer-based EEG Decoding: A Survey](https://mlsanity.com/paper/2507.02320/) (2025)
- [SingLEM: Single-Channel Large EEG Model](https://mlsanity.com/paper/2509.17920/) (2025)
- [Decoding Covert Speech from EEG Using a Functional Areas Spatio-Temporal Transformer](https://mlsanity.com/paper/2504.03762/) (2025)
- [CMCRD: Cross-Modal Contrastive Representation Distillation for Emotion Recognition](https://mlsanity.com/paper/2504.09221/) (2025)
- [BrainPro: Towards Large-scale Brain State-aware EEG Representation Learning](https://mlsanity.com/paper/2509.22050/) (2025)
- [Variational Mixture of Graph Neural Experts for Alzheimer's Disease Recognition across Frequency Bands in EEG Brain Networks](https://mlsanity.com/paper/2510.11917/) (2025)
- [DSAINet: An Efficient Dual-Scale Attentive Interaction Network for General EEG Decoding](https://mlsanity.com/paper/2604.18095/) (2026)
- [Multi-class Decoding of Attended Speaker Direction Using Electroencephalogram and Audio Spatial Spectrum](https://mlsanity.com/paper/2411.06928/) (2024)
- [Perception-Guided EEG Analysis: A Deep Learning Approach Inspired by Level of Detail (LOD) Theory](https://mlsanity.com/paper/2501.10428/) (2025)
- [Spiking Neural Network for Intra-cortical Brain Signal Decoding](https://mlsanity.com/paper/2504.09213/) (2025)
- [AFPM: Alignment-based Frame Patch Modeling for Cross-Dataset EEG Decoding](https://mlsanity.com/paper/2507.11911/) (2025)
- [DLGE: Dual Local-Global Encoding for Generalizable Cross-BCI-Paradigm](https://mlsanity.com/paper/2509.07991/) (2025)
- [EEG-based AI-BCI Wheelchair Advancement: Transformer-Based Learning with Motor Imagery for Brain Computer Interface](https://mlsanity.com/paper/2509.25667/) (2025)
- [RatioWaveNet: A Learnable RDWT Front-End for Robust and Interpretable EEG Motor-Imagery Classification](https://mlsanity.com/paper/2510.21841/) (2025)
- [Graph-Based Learning of Spectro-Topographical EEG Representations with Gradient Alignment for Brain-Computer Interfaces](https://mlsanity.com/paper/2512.07820/) (2025)
- [RAICL: Retrieval-Augmented In-Context Learning for Vision-Language-Model Based EEG Seizure Detection](https://mlsanity.com/paper/2601.17844/) (2026)
- [LI-DSN: A Layer-wise Interactive Dual-Stream Network for EEG Decoding](https://mlsanity.com/paper/2604.01889/) (2026)
- [DLink: Distilling Layer-wise and Dominant Knowledge from EEG Foundation Models](https://mlsanity.com/paper/2604.15016/) (2026)
- [ERP-XTTN: Interpretable Prototype-Guided Cross-Attention for Cross-Subject ERP Classification](https://mlsanity.com/paper/2606.02939/) (2026)
- [Beyond Flattened Tokens: Structure-Preserving EEG Decoding with Reusable TriDim Blocks](https://mlsanity.com/paper/2609.19842/) (2026)
- [Adaptive Forgetting for Nonstationary Optimization: Towards Robust EEG Decoding](https://mlsanity.com/paper/2609.24233/) (2026)
- [DEFT: Disentanglement-Enhanced Fine-Tuning for EEG Foundation Models](https://mlsanity.com/paper/2610.14497/) (2026)

## Abstract (arXiv)

Effectively learning the temporal dynamics in electroencephalogram (EEG) signals is challenging yet essential for decoding brain activities using brain-computer interfaces (BCIs). Although Transformers are popular for their long-term sequential learning ability in the BCI field, most methods combining Transformers with convolutional neural networks (CNNs) fail to capture the coarse-to-fine temporal dynamics of EEG signals. To overcome this limitation, we introduce EEG-Deformer, which incorporates two main novel components into a CNN-Transformer: (1) a Hierarchical Coarse-to-Fine Transformer (HCT) block that integrates a Fine-grained Temporal Learning (FTL) branch into Transformers, effectively discerning coarse-to-fine temporal patterns; and (2) a Dense Information Purification (DIP) module, which utilizes multi-level, purified temporal information to enhance decoding accuracy. Comprehensive experiments on three representative cognitive tasks-cognitive attention, driving fatigue, and mental workload detection-consistently confirm the generalizability of our proposed EEG-Deformer, demonstrating that it either outperforms or performs comparably to existing state-of-the-art methods. Visualization results show that EEG-Deformer learns from neurophysiologically meaningful brain regions for the corresponding cognitive tasks. The source code can be found at https://github.com/yi-ding-cs/EEG-Deformer.