An Approach to Simultaneous Acquisition of Real-Time MRI Video, EEG, and Surface EMG for Articulatory, Brain, and Muscle Activity During Speech Production
Not on the map. Why?
Abstract (arXiv)
Speech production is a complex process spanning neural planning, motor control, muscle activation, and articulatory kinematics. While the acoustic speech signal is the most accessible product of the speech production act, it does not directly reveal its causal neurophysiological substrates. We present the first simultaneous acquisition of real-time (dynamic) MRI, EEG, and surface EMG, capturing several key aspects of the speech production chain: brain signals, muscle activations, and articulatory movements. This multimodal acquisition paradigm presents substantial technical challenges, including MRI-induced electromagnetic interference and myogenic artifacts. To mitigate these, we introduce an artifact suppression pipeline tailored to this tri-modal setting. Once fully developed, this framework is poised to offer an unprecedented window into speech neuroscience and insights leading to brain-computer interface advances. The source code and data are available.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 22 of its 60 references that are in the map. First posted on arXiv 5 Mar 2026.
Builds on
22 of 60 references.
References · 22
- 2022Decoding speech perception from non-invasive brain recordingscited by 129
- 2023Towards Voice Reconstruction from EEG during Imagined Speechcited by 26
- 2021A multispeaker dataset of raw and reconstructed speech production real-time MRI video and 3D volumetric imagescited by 21
- 2024Coding Speech through Vocal Tract Kinematicscited by 18
- 2024Scaling Law in Neural Data: Non-Invasive Speech Decoding with 175 Hours of EEG Datacited by 13
- 2025Neural Codecs as Biosignal Tokenizerscited by 13
- 2024ConvConcatNet: a deep convolutional neural network to reconstruct mel spectrogram from the EEGcited by 12
- 2023Relating EEG to continuous speech using deep neural networks: a reviewcited by 11
- 2024Toward Fully-End-to-End Listened Speech Decoding from EEG Signalscited by 8
- 2025Enhancing Listened Speech Decoding from EEG via Parallel Phoneme Sequence Predictioncited by 6
- 2024Diff-ETS: Learning a Diffusion Probabilistic Model for Electromyography-to-Speech Conversioncited by 4
- 2025Articulatory Feature Prediction from Surface EMG during Speech Productioncited by 4
10 more
- 2025A Feasibility Study of Task-Based fMRI at 0.55 Tcited by 4
- 2025A Survey on Bridging EEG Signals and Generative AI: From Image and Text to Beyondcited by 3
- 2025Towards disentangling the contributions of articulation and acoustics in multimodal phoneme recognitioncited by 3
- 2025Interpretable Modeling of Articulatory Temporal Dynamics from real-time MRI for Phoneme Recognitioncited by 3
- 2025emg2speech: Synthesizing speech from electromyography using self-supervised speech modelscited by 3
- 2025On the Relationship between Accent Strength and Articulatory Featurescited by 2
- 2025A long-form single-speaker real-time MRI speech dataset and benchmarkcited by 2
- 2026Feasibility of simultaneous EEG-fMRI at 0.55 T: Recording, Denoising, and Functional Mappingcited by 2
- 2024Towards EMG-to-Speech with a Necklace Form Factorcited by 1
- 2025ARTI-6: Towards Six-dimensional Articulatory Speech Encodingcited by 1
Built on by
Citing papers per year: 2026: 2.
2 papers.