mlsanityData to 14 Oct 2026

Learning Interpretable Representations Leads to Semantically Faithful EEG-to-Text Generation

arXiv 2505.17099 ↗First posted 21 May 2025cs.CLcited by 5

Not on the map. Why?

Abstract (arXiv)

Pretrained generative models have opened new frontiers in brain decoding by enabling the synthesis of realistic texts and images from non-invasive brain recordings. However, the reliability of such outputs remains questionable--whether they truly reflect semantic activation in the brain, or are merely hallucinated by the powerful generative models. In this paper, we focus on EEG-to-text decoding and address its hallucination issue through the lens of posterior collapse. Acknowledging the underlying mismatch in information capacity between EEG and text, we reframe the decoding task as semantic summarization of core meanings rather than previously verbatim reconstruction of stimulus texts. To this end, we propose the Generative Language Inspection Model (GLIM), which emphasizes learning informative and interpretable EEG representations to improve semantic grounding under heterogeneous and small-scale data conditions. Experiments on the public ZuCo dataset demonstrate that GLIM consistently generates fluent, EEG-grounded sentences without teacher forcing. Moreover, it supports more robust evaluation beyond text similarity, through EEG-text retrieval and zero-shot semantic classification across sentiment categories, relation types, and corpus topics. Together, our architecture and evaluation protocols lay the foundation for reliable and scalable benchmarking in generative brain decoding.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 40 of its 62 references that are in the map. First posted on arXiv 21 May 2025.

Builds on

40 of 62 references.

References · 40

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  5. 2022Scalable Diffusion Models with Transformerscited by 6,054
  6. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  7. 2017Neural Discrete Representation Learningcited by 4,712
  8. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  9. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
  10. 2022Survey of Hallucination in Natural Language Generationcited by 2,075
  11. 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,729
  12. 2023A Survey on Multimodal Large Language Modelscited by 909
28 more
  1. 2024Hallucination of Multimodal Large Language Models: A Surveycited by 388
  2. 2023A Survey of Hallucination in Large Foundation Modelscited by 368
  3. 2025Layer by Layer: Uncovering Hidden Representations in Language Modelscited by 311
  4. 2023OneLLM: One Framework to Align All Modalities with Languagecited by 187
  5. 2022Seeing Beyond the Brain: Conditional Diffusion Model with Sparse Masked Modeling for Vision Decodingcited by 144
  6. 2022Decoding speech perception from non-invasive brain recordingscited by 130
  7. 2019Interpreting and improving natural-language processing (in machines) with natural language-processing (in the brain)cited by 128
  8. 2023Towards Large-scale 3D Representation Learning with Multi-dataset Point Prompt Trainingcited by 86
  9. 2021Open Vocabulary Electroencephalography-To-Text Decoding and Zero-shot Sentiment Classificationcited by 67
  10. 2019ZuCo 2.0: A Dataset of Physiological Recordings During Natural Reading and Annotationcited by 66
  11. 2023Brain decoding: toward real-time reconstruction of visual perceptioncited by 66
  12. 2017Z-Forcing: Training Stochastic Recurrent Networkscited by 51
  13. 2014Deep Neural Networks Rival the Representation of Primate IT Cortex for Core Visual Object Recognitioncited by 50
  14. 2015Transfer Learning in Brain-Computer Interfacescited by 39
  15. 2021Low-Dimensional Structure in the Space of Language Representations is Reflected in Brain Responsescited by 34
  16. 2024Are EEG-to-Text Models Working?cited by 32
  17. 2023Brain Diffusion for Visual Exploration: Cortical Discovery using Large Scale Generative Modelscited by 29
  18. 2022When classifying grammatical role, BERT doesn't care about word order... except when it matterscited by 24
  19. 2025Brain-to-Text Decoding: A Non-invasive Approach via Typingcited by 22
  20. 2024Estimating the Hallucination Rate of Generative AIcited by 20
  21. 2024Enhancing EEG-to-Text Decoding through Transferable Representations from Pre-trained Contrastive EEG-Text Masked Autoencodercited by 15
  22. 2024Spurious reconstruction from brain activitycited by 15
  23. 2021Inter-subject Deep Transfer Learning for Motor Imagery EEG Decodingcited by 12
  24. 2024Unveiling Thoughts: A Review of Advancements in EEG Brain Signal Decoding into Textcited by 10
  25. 2022ExtraPhrase: Efficient Data Augmentation for Abstractive Summarizationcited by 8
  26. 2024BrainBits: How Much of the Brain are Generative Reconstruction Methods Using?cited by 5
  27. 2025Pretraining Large Brain Language Model for Active BCI: Silent Speechcited by 4
  28. 2024When does word order matter and when doesn't it?cited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 5.

5 papers.