NeuroSonic: Conditional Flow Matching for EEG-to-Speech Reconstruction
On the map: Diffusion models → Medical imaging and segmentation
Abstract (arXiv)
Reconstructing continuous speech from scalp electroencephalography (EEG) remains fundamentally challenging. EEG provides a weak, spatially diffuse, and highly variable measurement of distributed cortical activity, whereas speech is organized as a coherent acoustic trajectory with strong harmonic and temporal structure. The resulting mismatch makes waveform regression unstable and causes stochastic multi-step generation to be sensitive to artifact-dependent conditioning and subject variability. We introduce NeuroSonic, a conditional flow-matching framework for EEG-to-speech reconstruction. Instead of predicting waveforms directly or refining them through stochastic denoising, NeuroSonic learns a deterministic probability-flow velocity field that transports a noise-corrupted acoustic state toward clean speech under EEG conditioning. EEG and audio are embedded into a shared token space and processed by a time-conditioned gated Transformer that parameterizes the transport ordinary differential equation. This formulation models trajectory evolution explicitly while avoiding iterative stochastic sampling. We evaluate NeuroSonic on the CineBrain and EAV benchmarks under cross-subject evaluation. Across both datasets, the proposed method improves distributional realism, spectral fidelity, and perceptual quality over representative GAN-, diffusion-, and mean-flow baselines, with up to a 26.3% gain in overall perceptual quality. The performance gap is most evident in artifact-heavy segments, where conditioning variability is strongest. These findings indicate that deterministic conditional transport provides a stable and effective formulation for EEG-driven speech reconstruction. Code is available at https://github.com/Y-Research-SBU/NeuroSonic/ .
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models. Builds on 20 of its 31 references that are in the map. First posted on arXiv 23 Jun 2026.
Builds on
20 of 31 references.
References · 20
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2022Flow Matching for Generative Modelingcited by 5,702
- 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,154
- 2025Mean Flows for One-step Generative Modelingcited by 561
- 2020DNSMOS: A Non-Intrusive Perceptual Objective Speech Quality metric to evaluate Noise Suppressorscited by 241
- 2023Mousai: Text-to-Music Generation with Long-Context Latent Diffusioncited by 78
- 2024EEGFormer: Towards Transferable and Interpretable Large-Scale EEG Foundation Modelcited by 52
- 2023MedGen3D: A Deep Generative Framework for Paired 3D Image and Mask Generationcited by 34
- 2024Calibrating Multi-modal Representations: A Pursuit of Group Robustness without Annotationscited by 30
- 2021Self-supervised Dialogue Learning for Spoken Conversational Question Answeringcited by 28
- 2023Towards Voice Reconstruction from EEG during Imagined Speechcited by 26
- 2020Adaptive Bi-directional Attention: Exploring Multi-Granularity Representations for Machine Reading Comprehensioncited by 22
8 more
- 2025Uncovering Memorization Effect in the Presence of Spurious Correlationscited by 20
- 2025Ouroboros: Single-step Diffusion Models for Cycle-consistent Forward and Inverse Renderingcited by 16
- 2022Aligning Source Visual and Target Language Domains for Unpaired Video Captioningcited by 14
- 2026Stable Velocity: A Variance Perspective on Flow Matchingcited by 12
- 2025CineBrain: A Large-Scale Multi-Modal Brain Dataset During Naturalistic Audiovisual Narrative Processingcited by 11
- 2026Let EEG Models Learn EEGcited by 7
- 2025Scale Where It Matters: Training-Free Localized Scaling for Diffusion Modelscited by 6
- 2026Inter- and Intra-Subject Variability in EEG: A Systematic Surveycited by 4
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026NEUROTOKEN: Joint Source and Directional AAD with Envelope Decoding via Conditional Flow Matching
- 2026B[FM]2: Brain Foundation Model via Flow Matching with SplitUNet
- 2026DenoFlow: Flow Matching for SSVEP Denoising under Real Physiological Artifacts
- 2026Brain-to-Speech: Prosody Feature Engineering and Transformer-Based Reconstruction
- 2025BrainStratify: Coarse-to-Fine Disentanglement of Intracranial Neural Dynamics
- 2023Brain-Driven Representation Learning Based on Diffusion Model
- 2026TRACE: Temporal Routing with Autoregressive Cross-channel Experts for EEG Representation Learning
- 2026Channel-Preserving Representation Alignment for EEG-to-Music Reconstruction