# Relating EEG to continuous speech using deep neural networks: a review

arXiv: https://arxiv.org/abs/2302.01736 · HTML: https://mlsanity.com/paper/2302.01736/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Feb 2023.

## Built on by (most cited first, 12 of 12)

- [Toward Fully-End-to-End Listened Speech Decoding from EEG Signals](https://mlsanity.com/paper/2406.08644/) (2024)
- [Enhancing Listened Speech Decoding from EEG via Parallel Phoneme Sequence Prediction](https://mlsanity.com/paper/2501.04844/) (2025)
- [DGSD: Dynamical Graph Self-Distillation for EEG-Based Auditory Spatial Attention Detection](https://mlsanity.com/paper/2309.07147/) (2023)
- [An Approach to Simultaneous Acquisition of Real-Time MRI Video, EEG, and Surface EMG for Articulatory, Brain, and Muscle Activity During Speech Production](https://mlsanity.com/paper/2603.04840/) (2026)
- [Multi-class Decoding of Attended Speaker Direction Using Electroencephalogram and Audio Spatial Spectrum](https://mlsanity.com/paper/2411.06928/) (2024)
- [S2M-Former: Spiking Symmetric Mixing Branchformer for Brain Auditory Attention Detection](https://mlsanity.com/paper/2508.05164/) (2025)
- [SHINE: Sequential Hierarchical Integration Network for EEG and MEG](https://mlsanity.com/paper/2602.23960/) (2026)
- [Post-processing of EEG-based Auditory Attention Decoding Decisions via Hidden Markov Models](https://mlsanity.com/paper/2506.24024/) (2025)
- [Scattering Transform for Auditory Attention Decoding](https://mlsanity.com/paper/2602.23003/) (2026)
- [Decoding Stimulus Reconstruction-Based Auditory Attention Robustly in Unbalanced EEG Datasets](https://mlsanity.com/paper/2605.25605/) (2026)
- [Addressing Limited Data in Auditory Attention Decoding with Diffusion Generative Models](https://mlsanity.com/paper/2607.18345/) (2026)
- [Merit: Making every modality earn its contribution in multimodal attention decoding](https://mlsanity.com/paper/2610.17507/) (2026)

## Abstract (arXiv)

Objective. When a person listens to continuous speech, a corresponding response is elicited in the brain and can be recorded using electroencephalography (EEG). Linear models are presently used to relate the EEG recording to the corresponding speech signal. The ability of linear models to find a mapping between these two signals is used as a measure of neural tracking of speech. Such models are limited as they assume linearity in the EEG-speech relationship, which omits the nonlinear dynamics of the brain. As an alternative, deep learning models have recently been used to relate EEG to continuous speech. Approach. This paper reviews and comments on deep-learning-based studies that relate EEG to continuous speech in single- or multiple-speakers paradigms. We point out recurrent methodological pitfalls and the need for a standard benchmark of model analysis. Main results. We gathered 29 studies. The main methodological issues we found are biased cross-validations, data leakage leading to over-fitted models, or disproportionate data size compared to the model's complexity. In addition, we address requirements for a standard benchmark model analysis, such as public datasets, common evaluation metrics, and good practices for the match-mismatch task. Significance. We present a review paper summarizing the main deep-learning-based studies that relate EEG to speech while addressing methodological pitfalls and important considerations for this newly expanding field. Our study is particularly relevant given the growing application of deep learning in EEG-speech decoding.