mlsanityData to 14 Oct 2026

Toward Fully-End-to-End Listened Speech Decoding from EEG Signals

arXiv 2406.08644 ↗First posted 12 Jun 2024eess.SP cs.AI cs.SDcited by 8

Not on the map. Why?

Abstract (arXiv)

Speech decoding from EEG signals is a challenging task, where brain activity is modeled to estimate salient characteristics of acoustic stimuli. We propose FESDE, a novel framework for Fully-End-to-end Speech Decoding from EEG signals. Our approach aims to directly reconstruct listened speech waveforms given EEG signals, where no intermediate acoustic feature processing step is required. The proposed method consists of an EEG module and a speech module along with a connector. The EEG module learns to better represent EEG signals, while the speech module generates speech waveforms from model representations. The connector learns to bridge the distributions of the latent spaces of EEG and speech. The proposed framework is both simple and efficient, by allowing single-step inference, and outperforms prior works on objective metrics. A fine-grained phoneme analysis is conducted to unveil model characteristics of speech decoding. The source code is available here: github.com/lee-jhwn/fesde.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 15 of its 29 references that are in the map. First posted on arXiv 12 Jun 2024.

Builds on

15 of 29 references.

References · 15

  1. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,465
  2. 2016Density estimation using Real NVPcited by 2,042
  3. 2016WaveNet: A Generative Model for Raw Audiocited by 1,722
  4. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,154
  5. 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 561
  6. 2018WaveGlow: A Flow-based Generative Network for Speech Synthesiscited by 309
  7. 2020Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Searchcited by 273
  8. 2021BENDR: using transformers and a contrastive self-supervised learning task to learn from massive amounts of EEG datacited by 199
  9. 2021A Survey on Neural Speech Synthesiscited by 184
  10. 2022Decoding speech perception from non-invasive brain recordingscited by 130
  11. 2023Towards Voice Reconstruction from EEG during Imagined Speechcited by 26
  12. 2023Relating EEG to continuous speech using deep neural networks: a reviewcited by 12
3 more
  1. 2024ConvConcatNet: a deep convolutional neural network to reconstruct mel spectrogram from the EEGcited by 12
  2. 2024Knowledge-guided EEG Representation Learningcited by 7
  3. 2023Neural2Speech: A Transfer Learning Framework for Neural-Driven Speech Reconstructioncited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 6; 2026: 2.

8 papers.

  1. 2025Neural Codecs as Biosignal Tokenizerscited by 14
  2. 2025Enhancing Listened Speech Decoding from EEG via Parallel Phoneme Sequence Predictioncited by 6
  3. 2025Articulatory Feature Prediction from Surface EMG during Speech Productioncited by 4
  4. 2025A Survey on Bridging EEG Signals and Generative AI: From Image and Text to Beyondcited by 3
  5. 2025EEG-ReMinD: Enhancing Neurodegenerative EEG Decoding through Self-Supervised State Reconstruction-Primed Riemannian Dynamicscited by 2
  6. 2026An Approach to Simultaneous Acquisition of Real-Time MRI Video, EEG, and Surface EMG for Articulatory, Brain, and Muscle Activity During Speech Productioncited by 2
  7. 2025Neuro2Semantic: A Transfer Learning Framework for Semantic Reconstruction of Continuous Language from Human Intracranial EEGcited by 1
  8. 2026MindVoice: Reconstructing Intelligible Speech from Non-invasive Neural Signals with Pretrained Priorscited by 0