# EEG2TEXT: Open Vocabulary EEG-to-Text Decoding with EEG Pre-Training and Multi-View Transformer

arXiv: https://arxiv.org/abs/2405.02165 · HTML: https://mlsanity.com/paper/2405.02165/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 31 references that are in the map.
- First posted on arXiv 3 May 2024.

## Builds on (most cited first, 9 of 9)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [SpanBERT: Improving Pre-training by Representing and Predicting Spans](https://mlsanity.com/paper/1907.10529/) (2019)
- [BENDR: using transformers and a contrastive self-supervised learning task to learn from massive amounts of EEG data](https://mlsanity.com/paper/2101.12037/) (2021)
- [BrainBERT: Self-supervised representation learning for intracranial recordings](https://mlsanity.com/paper/2302.14367/) (2023)
- [Open Vocabulary Electroencephalography-To-Text Decoding and Zero-shot Sentiment Classification](https://mlsanity.com/paper/2112.02690/) (2021)
- [MAEEG: Masked Auto-encoder for EEG Representation Learning](https://mlsanity.com/paper/2211.02625/) (2022)
- [DeWave: Discrete EEG Waves Encoding for Brain Dynamics to Text Translation](https://mlsanity.com/paper/2309.14030/) (2023)

## Built on by (most cited first, 20 of 20)

- [Thought2Text: Text Generation from EEG Signal using Large Language Models (LLMs)](https://mlsanity.com/paper/2410.07507/) (2024)
- [Tokenizing Single-Channel EEG with Time-Frequency Motif Learning](https://mlsanity.com/paper/2502.16060/) (2025)
- [Brain4FMs: A Benchmark of Foundation Models for Electrical Brain Signal](https://mlsanity.com/paper/2602.11558/) (2026)
- [SEE: Semantically Aligned EEG-to-Text Translation](https://mlsanity.com/paper/2409.16312/) (2024)
- [Enhancing Listened Speech Decoding from EEG via Parallel Phoneme Sequence Prediction](https://mlsanity.com/paper/2501.04844/) (2025)
- [Large Language Models for EEG: A Comprehensive Survey and Taxonomy](https://mlsanity.com/paper/2506.06353/) (2025)
- [Embracing Trustworthy Brain-Agent Collaboration as Paradigm Extension for Intelligent Assistive Technologies](https://mlsanity.com/paper/2510.22095/) (2025)
- [EEG2TEXT-CN: An Exploratory Study of Open-Vocabulary Chinese Text-EEG Alignment via Large Language Model and Contrastive Learning on ChineseEEG](https://mlsanity.com/paper/2506.00854/) (2025)
- [Foundation Models for Cross-Domain EEG Analysis Application: A Survey](https://mlsanity.com/paper/2508.15716/) (2025)
- [A Survey of Spatio-Temporal EEG data Analysis: from Models to Applications](https://mlsanity.com/paper/2410.08224/) (2024)
- [A Survey on Bridging EEG Signals and Generative AI: From Image and Text to Beyond](https://mlsanity.com/paper/2502.12048/) (2025)
- [Transformer-based EEG Decoding: A Survey](https://mlsanity.com/paper/2507.02320/) (2025)
- [How Many Bytes Can You Take Out Of Brain-To-Text Decoding?](https://mlsanity.com/paper/2405.14055/) (2024)
- [Decoding the Multimodal Mind: Generalizable Brain-to-Text Translation via Multimodal Alignment and Adaptive Routing](https://mlsanity.com/paper/2505.10356/) (2025)
- [EEG-to-Text Translation: A Model for Deciphering Human Brain Activity](https://mlsanity.com/paper/2505.13936/) (2025)
- [Neuro2Semantic: A Transfer Learning Framework for Semantic Reconstruction of Continuous Language from Human Intracranial EEG](https://mlsanity.com/paper/2506.00381/) (2025)
- [Neurocognitive Modeling for Text Generation: Deep Learning Architecture for EEG Data](https://mlsanity.com/paper/2509.07202/) (2025)
- [Hierarchic-EEG2Text: Assessing EEG-To-Text Decoding across Hierarchical Abstraction Levels](https://mlsanity.com/paper/2602.20932/) (2026)
- [Brain-to-Language Decoding: Tasks, Signals, Methods, Evaluation, Practical Use and Beyond](https://mlsanity.com/paper/2609.27650/) (2026)
- [Joint Text-Audio Alignment for EEG-to-Text Decoding in Chinese Speech Production and Perception](https://mlsanity.com/paper/2607.25626/) (2026)

## Abstract (arXiv)

Deciphering the intricacies of the human brain has captivated curiosity for centuries. Recent strides in Brain-Computer Interface (BCI) technology, particularly using motor imagery, have restored motor functions such as reaching, grasping, and walking in paralyzed individuals. However, unraveling natural language from brain signals remains a formidable challenge. Electroencephalography (EEG) is a non-invasive technique used to record electrical activity in the brain by placing electrodes on the scalp. Previous studies of EEG-to-text decoding have achieved high accuracy on small closed vocabularies, but still fall short of high accuracy when dealing with large open vocabularies. We propose a novel method, EEG2TEXT, to improve the accuracy of open vocabulary EEG-to-text decoding. Specifically, EEG2TEXT leverages EEG pre-training to enhance the learning of semantics from EEG signals and proposes a multi-view transformer to model the EEG signal processing by different spatial regions of the brain. Experiments show that EEG2TEXT has superior performance, outperforming the state-of-the-art baseline methods by a large margin of up to 5% in absolute BLEU and ROUGE scores. EEG2TEXT shows great potential for a high-performance open-vocabulary brain-to-text system to facilitate communication.