# Fusing Pretrained ViTs with TCNet for Enhanced EEG Regression

arXiv: https://arxiv.org/abs/2404.15311 · HTML: https://mlsanity.com/paper/2404.15311/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 56 references that are in the map.
- First posted on arXiv 2 Apr 2024.

## Builds on (most cited first, 18 of 18)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling](https://mlsanity.com/paper/1803.01271/) (2018)
- [A Survey on Visual Transformer](https://mlsanity.com/paper/2012.12556/) (2020)
- [MS-TCN: Multi-Stage Temporal Convolutional Network for Action Segmentation](https://mlsanity.com/paper/1903.01945/) (2019)
- [Visual Transformers: Token-based Image Representation and Processing for Computer Vision](https://mlsanity.com/paper/2006.03677/) (2020)
- [Deep learning-based electroencephalography analysis: a systematic review](https://mlsanity.com/paper/1901.05498/) (2019)
- [Machine Learning for Synthetic Data Generation: A Review](https://mlsanity.com/paper/2302.04062/) (2023)
- [EEG-TCNet: An Accurate Temporal Convolutional Network for Embedded Motor-Imagery Brain-Machine Interfaces](https://mlsanity.com/paper/2006.00622/) (2020)
- [Machine Learning for Predicting Epileptic Seizures Using EEG Signals: A Review](https://mlsanity.com/paper/2002.01925/) (2020)
- [EEGEyeNet: a Simultaneous Electroencephalography and Eye-tracking Dataset and Benchmark for Eye Movement Prediction](https://mlsanity.com/paper/2111.05100/) (2021)
- [Transfer Learning for Human Activity Recognition using Representational Analysis of Neural Networks](https://mlsanity.com/paper/2012.04479/) (2020)
- [Successfully Applying Lottery Ticket Hypothesis to Diffusion Model](https://mlsanity.com/paper/2310.18823/) (2023)
- [Time Majority Voting, a PC-based EEG Classifier for Non-expert Users](https://mlsanity.com/paper/2207.12662/) (2022)
- [Trends in Machine Learning and Electroencephalogram (EEG): A Review for Undergraduate Researchers](https://mlsanity.com/paper/2307.02819/) (2023)
- [ViT2EEG: Leveraging Hybrid Pretrained Vision Transformers for EEG Data](https://mlsanity.com/paper/2308.00454/) (2023)
- [BrainActivity1: A Framework of EEG Data Collection and Machine Learning Analysis for College Students](https://mlsanity.com/paper/2207.13239/) (2022)
- [Spatial-temporal Transformers for EEG Emotion Recognition](https://mlsanity.com/paper/2110.06553/) (2021)

## Built on by (most cited first, 2 of 2)

- [EEGMobile: Enhancing Speed and Accuracy in EEG-Based Gaze Prediction with Advanced Mobile Architectures](https://mlsanity.com/paper/2408.03449/) (2024)
- [Bidirectional Temporal Dynamics Modeling for EEG-based Driving Fatigue Recognition](https://mlsanity.com/paper/2602.14071/) (2026)

## Abstract (arXiv)

The task of Electroencephalogram (EEG) analysis is paramount to the development of Brain-Computer Interfaces (BCIs). However, to reach the goal of developing robust, useful BCIs depends heavily on the speed and the accuracy at which BCIs can understand neural dynamics. In response to that goal, this paper details the integration of pre-trained Vision Transformers (ViTs) with Temporal Convolutional Networks (TCNet) to enhance the precision of EEG regression. The core of this approach lies in harnessing the sequential data processing strengths of ViTs along with the superior feature extraction capabilities of TCNet, to significantly improve EEG analysis accuracy. In addition, we analyze the importance of how to construct optimal patches for the attention mechanism to analyze, balancing both speed and accuracy tradeoffs. Our results showcase a substantial improvement in regression accuracy, as evidenced by the reduction of Root Mean Square Error (RMSE) from 55.4 to 51.8 on EEGEyeNet's Absolute Position Task, outperforming existing state-of-the-art models. Without sacrificing performance, we increase the speed of this model by an order of magnitude (up to 4.32x faster). This breakthrough not only sets a new benchmark in EEG regression analysis but also opens new avenues for future research in the integration of transformer architectures with specialized feature extraction methods for diverse EEG datasets.