# Diff-E: Diffusion-based Learning for Decoding Imagined Speech EEG

arXiv: https://arxiv.org/abs/2307.14389 · HTML: https://mlsanity.com/paper/2307.14389/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models.
- Builds on 14 of its 35 references that are in the map.
- First posted on arXiv 26 Jul 2023.

## Placed in

- Diffusion models / [Medical imaging and segmentation](https://mlsanity.com/diffusion/medical-imaging-and-segmentation/)

## Builds on (most cited first, 14 of 14)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Cyclical Learning Rates for Training Neural Networks](https://mlsanity.com/paper/1506.01186/) (2015)
- [EEGNet: A Compact Convolutional Network for EEG-based Brain-Computer Interfaces](https://mlsanity.com/paper/1611.08024/) (2016)
- [CSDI: Conditional Score-based Diffusion Models for Probabilistic Time Series Imputation](https://mlsanity.com/paper/2107.03502/) (2021)
- [Diffusion Autoencoders: Toward a Meaningful and Decodable Representation](https://mlsanity.com/paper/2111.15640/) (2021)
- [Autoregressive Denoising Diffusion Models for Multivariate Probabilistic Time Series Forecasting](https://mlsanity.com/paper/2101.12072/) (2021)
- [Diffusion-based Time Series Imputation and Forecasting with Structured State Space Models](https://mlsanity.com/paper/2208.09399/) (2022)
- [Diff-TTS: A Denoising Diffusion Model for Text-to-Speech](https://mlsanity.com/paper/2104.01409/) (2021)
- [Unsupervised Representation Learning from Pre-trained Diffusion Probabilistic Models](https://mlsanity.com/paper/2212.12990/) (2022)
- [Towards Voice Reconstruction from EEG during Imagined Speech](https://mlsanity.com/paper/2301.07173/) (2023)
- [EEG-Transformer: Self-attention from Transformer Architecture for Decoding EEG of Imagined Speech](https://mlsanity.com/paper/2112.09239/) (2021)

## Built on by (most cited first, 17 of 17)

- [A Survey on Diffusion Models for Time Series and Spatio-Temporal Data](https://mlsanity.com/paper/2404.18886/) (2024)
- [RespDiff: An End-to-End Multi-scale RNN Diffusion Model for Respiratory Waveform Estimation from PPG Signals](https://mlsanity.com/paper/2410.04366/) (2024)
- [Diff-ETS: Learning a Diffusion Probabilistic Model for Electromyography-to-Speech Conversion](https://mlsanity.com/paper/2405.08021/) (2024)
- [D3A-TS: Denoising-Driven Data Augmentation in Time Series](https://mlsanity.com/paper/2312.05550/) (2023)
- [Towards Dynamic Neural Communication and Speech Neuroprosthesis Based on Viseme Decoding](https://mlsanity.com/paper/2501.14790/) (2025)
- [Decoding Covert Speech from EEG Using a Functional Areas Spatio-Temporal Transformer](https://mlsanity.com/paper/2504.03762/) (2025)
- [Towards Unified Neural Decoding of Perceived, Spoken and Imagined Speech from EEG Signals](https://mlsanity.com/paper/2411.09243/) (2024)
- [EEGDM: Learning EEG Representation with Latent Diffusion Model](https://mlsanity.com/paper/2508.20705/) (2025)
- [MultiDiffNet: A Multi-Objective Diffusion Framework for Generalizable Brain Decoding](https://mlsanity.com/paper/2511.18294/) (2025)
- [Neural Speech Embeddings for Speech Synthesis Based on Deep Generative Networks](https://mlsanity.com/paper/2312.05814/) (2023)
- [An Audio-textual Diffusion Model For Converting Speech Signals Into Ultrasound Tongue Imaging Data](https://mlsanity.com/paper/2403.05820/) (2024)
- [Towards Scalable Handwriting Communication via EEG Decoding and Latent Embedding Integration](https://mlsanity.com/paper/2411.09170/) (2024)
- [Dynamic Neural Communication: Convergence of Computer Vision and Brain-Computer Interface](https://mlsanity.com/paper/2411.09211/) (2024)
- [Reconstructing Unseen Sentences from Speech-related Biosignals for Open-vocabulary Neural Communication](https://mlsanity.com/paper/2510.27247/) (2025)
- [Distinct Theta Synchrony across Speech Modes: Perceived, Spoken, Whispered, and Imagined](https://mlsanity.com/paper/2511.07918/) (2025)
- [Lightweight Diffusion-based Framework for Online Imagined Speech Decoding in Aphasia](https://mlsanity.com/paper/2511.07920/) (2025)
- [Synthetic Data Generation for Brain-Computer Interfaces: Overview, Benchmarking, and Future Directions](https://mlsanity.com/paper/2603.12296/) (2026)

## Abstract (arXiv)

Decoding EEG signals for imagined speech is a challenging task due to the high-dimensional nature of the data and low signal-to-noise ratio. In recent years, denoising diffusion probabilistic models (DDPMs) have emerged as promising approaches for representation learning in various domains. Our study proposes a novel method for decoding EEG signals for imagined speech using DDPMs and a conditional autoencoder named Diff-E. Results indicate that Diff-E significantly improves the accuracy of decoding EEG signals for imagined speech compared to traditional machine learning techniques and baseline models. Our findings suggest that DDPMs can be an effective tool for EEG signal decoding, with potential implications for the development of brain-computer interfaces that enable communication through imagined speech.