Learning Interpretable Representations Leads to Semantically Faithful EEG-to-Text Generation
Not on the map. Why?
Abstract (arXiv)
Pretrained generative models have opened new frontiers in brain decoding by enabling the synthesis of realistic texts and images from non-invasive brain recordings. However, the reliability of such outputs remains questionable--whether they truly reflect semantic activation in the brain, or are merely hallucinated by the powerful generative models. In this paper, we focus on EEG-to-text decoding and address its hallucination issue through the lens of posterior collapse. Acknowledging the underlying mismatch in information capacity between EEG and text, we reframe the decoding task as semantic summarization of core meanings rather than previously verbatim reconstruction of stimulus texts. To this end, we propose the Generative Language Inspection Model (GLIM), which emphasizes learning informative and interpretable EEG representations to improve semantic grounding under heterogeneous and small-scale data conditions. Experiments on the public ZuCo dataset demonstrate that GLIM consistently generates fluent, EEG-grounded sentences without teacher forcing. Moreover, it supports more robust evaluation beyond text similarity, through EEG-text retrieval and zero-shot semantic classification across sentiment categories, relation types, and corpus topics. Together, our architecture and evaluation protocols lay the foundation for reliable and scalable benchmarking in generative brain decoding.
Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 40 of its 62 references that are in the map. First posted on arXiv 21 May 2025.
Builds on
40 of 62 references.
References · 40
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
- 2022Scalable Diffusion Models with Transformerscited by 6,054
- 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
- 2017Neural Discrete Representation Learningcited by 4,712
- 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
- 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
- 2022Survey of Hallucination in Natural Language Generationcited by 2,075
- 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,729
- 2023A Survey on Multimodal Large Language Modelscited by 909
28 more
- 2024Hallucination of Multimodal Large Language Models: A Surveycited by 388
- 2023A Survey of Hallucination in Large Foundation Modelscited by 368
- 2025Layer by Layer: Uncovering Hidden Representations in Language Modelscited by 311
- 2023OneLLM: One Framework to Align All Modalities with Languagecited by 187
- 2022Seeing Beyond the Brain: Conditional Diffusion Model with Sparse Masked Modeling for Vision Decodingcited by 144
- 2022Decoding speech perception from non-invasive brain recordingscited by 130
- 2019Interpreting and improving natural-language processing (in machines) with natural language-processing (in the brain)cited by 128
- 2023Towards Large-scale 3D Representation Learning with Multi-dataset Point Prompt Trainingcited by 86
- 2021Open Vocabulary Electroencephalography-To-Text Decoding and Zero-shot Sentiment Classificationcited by 67
- 2019ZuCo 2.0: A Dataset of Physiological Recordings During Natural Reading and Annotationcited by 66
- 2023Brain decoding: toward real-time reconstruction of visual perceptioncited by 66
- 2017Z-Forcing: Training Stochastic Recurrent Networkscited by 51
- 2014Deep Neural Networks Rival the Representation of Primate IT Cortex for Core Visual Object Recognitioncited by 50
- 2015Transfer Learning in Brain-Computer Interfacescited by 39
- 2021Low-Dimensional Structure in the Space of Language Representations is Reflected in Brain Responsescited by 34
- 2024Are EEG-to-Text Models Working?cited by 32
- 2023Brain Diffusion for Visual Exploration: Cortical Discovery using Large Scale Generative Modelscited by 29
- 2022When classifying grammatical role, BERT doesn't care about word order... except when it matterscited by 24
- 2025Brain-to-Text Decoding: A Non-invasive Approach via Typingcited by 22
- 2024Estimating the Hallucination Rate of Generative AIcited by 20
- 2024Enhancing EEG-to-Text Decoding through Transferable Representations from Pre-trained Contrastive EEG-Text Masked Autoencodercited by 15
- 2024Spurious reconstruction from brain activitycited by 15
- 2021Inter-subject Deep Transfer Learning for Motor Imagery EEG Decodingcited by 12
- 2024Unveiling Thoughts: A Review of Advancements in EEG Brain Signal Decoding into Textcited by 10
- 2022ExtraPhrase: Efficient Data Augmentation for Abstractive Summarizationcited by 8
- 2024BrainBits: How Much of the Brain are Generative Reconstruction Methods Using?cited by 5
- 2025Pretraining Large Brain Language Model for Active BCI: Silent Speechcited by 4
- 2024When does word order matter and when doesn't it?cited by 1
Built on by
Citing papers per year: 2026: 5.
5 papers.
- 2026SENSE: Efficient EEG-to-Text via Privacy-Preserving Semantic Retrievalcited by 2
- 2026Hierarchic-EEG2Text: Assessing EEG-To-Text Decoding across Hierarchical Abstraction Levelscited by 1
- 2026Escaping the BLEU Trap: A Signal-Grounded Framework with Decoupled Semantic Guidance for EEG-to-Text Decodingcited by 1
- 2026Brain-to-Language Decoding: Tasks, Signals, Methods, Evaluation, Practical Use and Beyondcited by 1
- 2026SYNAPSE: Neuro-Symbolic Visual Thought-to-Text Decoding via Topological Semantic Denoisingcited by 0
- 2026Brain-to-Language Decoding: Tasks, Signals, Methods, Evaluation, Practical Use and Beyond23 Sep 2026
- 2026SYNAPSE: Neuro-Symbolic Visual Thought-to-Text Decoding via Topological Semantic Denoising27 May 2026
- 2026SENSE: Efficient EEG-to-Text via Privacy-Preserving Semantic Retrieval17 Mar 2026
- 2026Hierarchic-EEG2Text: Assessing EEG-To-Text Decoding across Hierarchical Abstraction Levels24 Feb 2026
- 2026Escaping the BLEU Trap: A Signal-Grounded Framework with Decoupled Semantic Guidance for EEG-to-Text Decoding9 Feb 2026