# Digital Voicing of Silent Speech

arXiv: https://arxiv.org/abs/2010.02960 · HTML: https://mlsanity.com/paper/2010.02960/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Oct 2020.

## Built on by (most cited first, 24 of 24)

- [An Improved Model for Voicing Silent Speech](https://mlsanity.com/paper/2106.01933/) (2021)
- [Twins-PainViT: Towards a Modality-Agnostic Vision Transformer Framework for Multimodal Automatic Pain Assessment using Facial Videos and fNIRS](https://mlsanity.com/paper/2407.19809/) (2024)
- [PainFormer: a Vision Foundation Model for Automatic Pain Assessment](https://mlsanity.com/paper/2505.01571/) (2025)
- [A Cross-Modal Approach to Silent Speech with LLM-Enhanced Recognition](https://mlsanity.com/paper/2403.05583/) (2024)
- [ASHA: Assistive Teleoperation via Human-in-the-Loop Reinforcement Learning](https://mlsanity.com/paper/2202.02465/) (2022)
- [Tiny-BioMoE: a Lightweight Embedding Model for Biosignal Analysis](https://mlsanity.com/paper/2507.21875/) (2025)
- [TinyMyo: a Tiny Foundation Model for Flexible EMG Signal Processing at the Edge](https://mlsanity.com/paper/2512.15729/) (2025)
- [Can LLMs Understand Unvoiced Speech? Exploring EMG-to-Text Conversion with LLMs](https://mlsanity.com/paper/2506.00304/) (2025)
- [Diff-ETS: Learning a Diffusion Probabilistic Model for Electromyography-to-Speech Conversion](https://mlsanity.com/paper/2405.08021/) (2024)
- [Decoding EEG Speech Perception with Transformers and VAE-based Data Augmentation](https://mlsanity.com/paper/2501.04359/) (2025)
- [Articulatory Feature Prediction from Surface EMG during Speech Production](https://mlsanity.com/paper/2505.13814/) (2025)
- [emg2speech: Synthesizing speech from electromyography using self-supervised speech models](https://mlsanity.com/paper/2510.23969/) (2025)
- [Towards Dynamic Neural Communication and Speech Neuroprosthesis Based on Viseme Decoding](https://mlsanity.com/paper/2501.14790/) (2025)
- [EPG2S: Speech Generation and Speech Enhancement based on Electropalatography and Audio Signals using Multimodal Learning](https://mlsanity.com/paper/2206.07860/) (2022)
- [Geometry of orofacial neuromuscular signals: speech articulation decoding using surface electromyography](https://mlsanity.com/paper/2411.02591/) (2024)
- [Affect Decoding in Phonated and Silent Speech Production from Surface EMG](https://mlsanity.com/paper/2603.11715/) (2026)
- [Comparison of sEMG Encoding Accuracy Across Speech Modes Using Articulatory and Phoneme Features](https://mlsanity.com/paper/2604.18920/) (2026)
- [A Silent Speech Decoding System from EEG and EMG with Heterogenous Electrode Configurations](https://mlsanity.com/paper/2506.13835/) (2025)
- [Reconstructing Unseen Sentences from Speech-related Biosignals for Open-vocabulary Neural Communication](https://mlsanity.com/paper/2510.27247/) (2025)
- [Cross-Modal Masking for Robust Silent Speech Synthesis Using sEMG and Lipreading](https://mlsanity.com/paper/2606.09667/) (2026)
- [SoniSpeech: A Large-Scale Open-Vocabulary Tri-Modal Dataset for Wearable Silent Speech Interfaces](https://mlsanity.com/paper/2608.00803/) (2026)
- [Myovox: Reading Speech from the Muscles of the Face](https://mlsanity.com/paper/2609.17548/) (2026)
- [Multi-Subject Pretraining Enables Short-Calibration Personalization for Closed-Corpus Surface EMG Speech Decoding](https://mlsanity.com/paper/2609.21288/) (2026)
- [MyoCodec: A Streaming Neural Codec for Electromyography](https://mlsanity.com/paper/2609.36687/) (2026)

## Abstract (arXiv)

In this paper, we consider the task of digitally voicing silent speech, where silently mouthed words are converted to audible speech based on electromyography (EMG) sensor measurements that capture muscle impulses. While prior work has focused on training speech synthesis models from EMG collected during vocalized speech, we are the first to train from EMG collected during silently articulated speech. We introduce a method of training on silent EMG by transferring audio targets from vocalized to silent signals. Our method greatly improves intelligibility of audio generated from silent EMG compared to a baseline that only trains with vocalized data, decreasing transcription word error rate from 64% to 4% in one data condition and 88% to 68% in another. To spur further development on this task, we share our new dataset of silent and vocalized facial EMG measurements.