# An Improved Model for Voicing Silent Speech

arXiv: https://arxiv.org/abs/2106.01933 · HTML: https://mlsanity.com/paper/2106.01933/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 21 references that are in the map.
- First posted on arXiv 3 Jun 2021.

## Builds on (most cited first, 7 of 7)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016)
- [Self-Attention with Relative Position Representations](https://mlsanity.com/paper/1803.02155/) (2018)
- [Deep Speech: Scaling up end-to-end speech recognition](https://mlsanity.com/paper/1412.5567/) (2014)
- [Wav2Letter: an End-to-End ConvNet-based Speech Recognition System](https://mlsanity.com/paper/1609.03193/) (2016)
- [Digital Voicing of Silent Speech](https://mlsanity.com/paper/2010.02960/) (2020)

## Built on by (most cited first, 15 of 15)

- [A Cross-Modal Approach to Silent Speech with LLM-Enhanced Recognition](https://mlsanity.com/paper/2403.05583/) (2024)
- [TinyMyo: a Tiny Foundation Model for Flexible EMG Signal Processing at the Edge](https://mlsanity.com/paper/2512.15729/) (2025)
- [Can LLMs Understand Unvoiced Speech? Exploring EMG-to-Text Conversion with LLMs](https://mlsanity.com/paper/2506.00304/) (2025)
- [Diff-ETS: Learning a Diffusion Probabilistic Model for Electromyography-to-Speech Conversion](https://mlsanity.com/paper/2405.08021/) (2024)
- [Articulatory Feature Prediction from Surface EMG during Speech Production](https://mlsanity.com/paper/2505.13814/) (2025)
- [Multimodal Segmentation for Vocal Tract Modeling](https://mlsanity.com/paper/2406.15754/) (2024)
- [emg2speech: Synthesizing speech from electromyography using self-supervised speech models](https://mlsanity.com/paper/2510.23969/) (2025)
- [Geometry of orofacial neuromuscular signals: speech articulation decoding using surface electromyography](https://mlsanity.com/paper/2411.02591/) (2024)
- [A Silent Speech Decoding System from EEG and EMG with Heterogenous Electrode Configurations](https://mlsanity.com/paper/2506.13835/) (2025)
- [CIS-BWE: Chaos-Informed Speech Bandwidth Extension](https://mlsanity.com/paper/2507.15970/) (2025)
- [Reconstructing Unseen Sentences from Speech-related Biosignals for Open-vocabulary Neural Communication](https://mlsanity.com/paper/2510.27247/) (2025)
- [SoniSpeech: A Large-Scale Open-Vocabulary Tri-Modal Dataset for Wearable Silent Speech Interfaces](https://mlsanity.com/paper/2608.00803/) (2026)
- [Myovox: Reading Speech from the Muscles of the Face](https://mlsanity.com/paper/2609.17548/) (2026)
- [Multi-Subject Pretraining Enables Short-Calibration Personalization for Closed-Corpus Surface EMG Speech Decoding](https://mlsanity.com/paper/2609.21288/) (2026)
- [MyoCodec: A Streaming Neural Codec for Electromyography](https://mlsanity.com/paper/2609.36687/) (2026)

## Abstract (arXiv)

In this paper, we present an improved model for voicing silent speech, where audio is synthesized from facial electromyography (EMG) signals. To give our model greater flexibility to learn its own input features, we directly use EMG signals as input in the place of hand-designed features used by prior work. Our model uses convolutional layers to extract features from the signals and Transformer layers to propagate information across longer distances. To provide better signal for learning, we also introduce an auxiliary task of predicting phoneme labels in addition to predicting speech audio features. On an open vocabulary intelligibility evaluation, our model improves the state of the art for this task by an absolute 25.8%.