# Efficacy of Transformer Networks for Classification of Raw EEG Data

arXiv: https://arxiv.org/abs/2202.05170 · HTML: https://mlsanity.com/paper/2202.05170/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 47 references that are in the map.
- First posted on arXiv 8 Feb 2022.

## Builds on (most cited first, 8 of 8)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity](https://mlsanity.com/paper/2101.03961/) (2021)
- [EEGNet: A Compact Convolutional Network for EEG-based Brain-Computer Interfaces](https://mlsanity.com/paper/1611.08024/) (2016)
- [BENDR: using transformers and a contrastive self-supervised learning task to learn from massive amounts of EEG data](https://mlsanity.com/paper/2101.12037/) (2021)
- [A Multimodal Approach to Estimating Vigilance Using EEG and Forehead EOG](https://mlsanity.com/paper/1611.08492/) (2016)
- [EEG based Continuous Speech Recognition using Transformers](https://mlsanity.com/paper/2001.00501/) (2019)

## Built on by (most cited first, 9 of 9)

- [Trends in Machine Learning and Electroencephalogram (EEG): A Review for Undergraduate Researchers](https://mlsanity.com/paper/2307.02819/) (2023)
- [ScatterFormer: Locally-Invariant Scattering Transformer for Patient-Independent Multispectral Detection of Epileptiform Discharges](https://mlsanity.com/paper/2304.14919/) (2023)
- [MVTN: A Multiscale Video Transformer Network for Hand Gesture Recognition](https://mlsanity.com/paper/2409.03890/) (2024)
- [Neurocognitive Modeling for Text Generation: Deep Learning Architecture for EEG Data](https://mlsanity.com/paper/2509.07202/) (2025)
- [Modified TSception for Analyzing Driver Drowsiness and Mental Workload from EEG](https://mlsanity.com/paper/2512.21747/) (2025)
- [Sample Dominance Aware Framework via Non-Parametric Estimation for Spontaneous Brain-Computer Interface](https://mlsanity.com/paper/2311.07079/) (2023)
- [On the Role of Activation Functions in EEG-To-Text Decoder](https://mlsanity.com/paper/2410.12572/) (2024)
- [HCFT: Hierarchical Convolutional Fusion Transformer for EEG Decoding](https://mlsanity.com/paper/2601.12279/) (2026)
- [Transformer Based Model for Spatiotemporal Feature Learning in EEG Emotion Recognition](https://mlsanity.com/paper/2606.10718/) (2026)

## Abstract (arXiv)

With the unprecedented success of transformer networks in natural language processing (NLP), recently, they have been successfully adapted to areas like computer vision, generative adversarial networks (GAN), and reinforcement learning. Classifying electroencephalogram (EEG) data has been challenging and researchers have been overly dependent on pre-processing and hand-crafted feature extraction. Despite having achieved automated feature extraction in several other domains, deep learning has not yet been accomplished for EEG. In this paper, the efficacy of the transformer network for the classification of raw EEG data (cleaned and pre-processed) is explored. The performance of transformer networks was evaluated on a local (age and gender data) and a public dataset (STEW). First, a classifier using a transformer network is built to classify the age and gender of a person with raw resting-state EEG data. Second, the classifier is tuned for mental workload classification with open access raw multi-tasking mental workload EEG data (STEW). The network achieves an accuracy comparable to state-of-the-art accuracy on both the local (Age and Gender dataset; 94.53% (gender) and 87.79% (age)) and the public (STEW dataset; 95.28% (two workload levels) and 88.72% (three workload levels)) dataset. The accuracy values have been achieved using raw EEG data without feature extraction. Results indicate that the transformer-based deep learning models can successfully abate the need for heavy feature-extraction of EEG data for successful classification.