mlsanityData to 14 Oct 2026

Thought2Text: Text Generation from EEG Signal using Large Language Models (LLMs)

arXiv 2410.07507 ↗First posted 10 Oct 2024cs.CLcited by 18

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Decoding and expressing brain activity in a comprehensible form is a challenging frontier in AI. This paper presents Thought2Text, which uses instruction-tuned Large Language Models (LLMs) fine-tuned with EEG data to achieve this goal. The approach involves three stages: (1) training an EEG encoder for visual feature extraction, (2) fine-tuning LLMs on image and text data, enabling multimodal description generation, and (3) further fine-tuning on EEG embeddings to generate text directly from EEG during inference. Experiments on a public EEG dataset collected for six subjects with image stimuli and text captions demonstrate the efficacy of multimodal LLMs (LLaMA-v3, Mistral-v0.3, Qwen2.5), validated using traditional language generation evaluation metrics, as well as fluency and adequacy measures. This approach marks a significant advancement towards portable, low-cost "thoughts-to-text" technology with potential applications in both neuroscience and natural language processing.

Key facts. Cited by 18 papers in the map (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 15 of its 45 references that are in the map. First posted on arXiv 10 Oct 2024.

Builds on

15 of 45 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023LLaMA: Open and Efficient Foundation Language Models27 Feb

Other references · 14

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  3. 2023Qwen Technical Reportcited by 3,344
  4. 2023Mistral 7Bcited by 3,164
  5. 2023Gemini: A Family of Highly Capable Multimodal Modelscited by 872
  6. 2023Textbooks Are All You Needcited by 569
  7. 2023AudioPaLM: A Large Language Model That Can Speak and Listencited by 343
  8. 2019ZuCo 2.0: A Dataset of Physiological Recordings During Natural Reading and Annotationcited by 66
  9. 2023Brain decoding: toward real-time reconstruction of visual perceptioncited by 66
  10. 2017Relevance of Unsupervised Metrics in Task-Oriented Dialogue for Evaluating Natural Language Generationcited by 61
  11. 2023DreamDiffusion: Generating High-Quality Images from Brain EEG Signalscited by 56
  12. 2024EEG2TEXT: Open Vocabulary EEG-to-Text Decoding with EEG Pre-Training and Multi-View Transformercited by 20
2 more
  1. 2022MindBigData 2022 A Large Dataset of Brain Signalscited by 7
  2. 2024EIT-1M: One Million EEG-Image-Text Pairs for Human Visual-textual Recognition and Morecited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 7; 2026: 10.

18 papers.

Sort:

By area:

  1. 2024A Combined Encoder and Transformer Approach for Coherent and High-Quality Text Generationcited by 7
  2. 2026LLM as Clinical Graph Structure Refiner: Enhancing Representation Learning in EEG Seizure Diagnosiscited by 7
  3. 2025WaveMind: Towards a Conversational EEG Foundation Model Aligned to Textual and Visual Modalitiescited by 6
  4. 2025Large Language Models for EEG: A Comprehensive Survey and Taxonomycited by 5
  5. 2025Embracing Trustworthy Brain-Agent Collaboration as Paradigm Extension for Intelligent Assistive Technologiescited by 5
  6. 2025Foundation Models for Cross-Domain EEG Analysis Application: A Surveycited by 4
  7. 2025A Survey on Bridging EEG Signals and Generative AI: From Image and Text to Beyondcited by 3
  8. 2026SENSE: Efficient EEG-to-Text via Privacy-Preserving Semantic Retrievalcited by 2
  9. 2025DELTA: Language Diffusion-based EEG-to-Text Architecturecited by 1
  10. 2026Escaping the BLEU Trap: A Signal-Grounded Framework with Decoupled Semantic Guidance for EEG-to-Text Decodingcited by 1
  11. 2026RAG-based EEG-to-Text Translation Using Deep Learning and LLMscited by 1
  12. 2026Margin-Regularized Structured Semantic Alignment for Brain-Language Correspondencecited by 1
6 more
  1. 2025Decoding Open-Ended Information Seeking Goals from Eye Movements in Readingcited by 0
  2. 2026Synthetic Data Generation for Brain-Computer Interfaces: Overview, Benchmarking, and Future Directionscited by 0
  3. 2026The Deep-Match Framework for Event-Related Potential Detection in EEGcited by 0
  4. 2026MindAlign: Decoding Inner Speech from fMRI Signals via Multimodal Embedding Alignment under Limited Datacited by 0
  5. 2026Large language models in medical time series analysiscited by 0
  6. 2026BAT-CLIP: Trimodal Alignment of Brain, Audio and Textcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025EEG2TEXT-CN: An Exploratory Study of Open-Vocabulary Chinese Text-EEG Alignment via Large Language Model and Contrastive Learning on ChineseEEG
  2. 2024BELT-2: Bootstrapping EEG-to-Language representation alignment for multi-task brain decoding
  3. 2025Interpretable EEG-to-Image Generation with Semantic Prompts
  4. 2025LLMs Help Alleviate the Cross-Subject Variability in Brain Signal and Language Alignment
  5. 2026Multimodal LLMs Can Learn to Read Brain Signals: A Vision--Language Model for Unified Multi-Task EEG Decoding
  6. 2025Decoding the Multimodal Mind: Generalizable Brain-to-Text Translation via Multimodal Alignment and Adaptive Routing
  7. 2025CATVis: Context-Aware Thought Visualization
  8. 2024EEG-Language Pretraining for Highly Label-Efficient Clinical Phenotyping