# Diff-ETS: Learning a Diffusion Probabilistic Model for Electromyography-to-Speech Conversion

arXiv: https://arxiv.org/abs/2405.08021 · HTML: https://mlsanity.com/paper/2405.08021/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Audio, speech and music, one of 14 directions in Diffusion models.
- Builds on 11 of its 22 references that are in the map.
- First posted on arXiv 11 May 2024.

## Placed in

- Diffusion models / [Audio, speech and music](https://mlsanity.com/diffusion/audio-speech-and-music/)

## Builds on (most cited first, 11 of 11)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis](https://mlsanity.com/paper/2010.05646/) (2020)
- [Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speech](https://mlsanity.com/paper/2105.06337/) (2021)
- [Diff-TTS: A Denoising Diffusion Model for Text-to-Speech](https://mlsanity.com/paper/2104.01409/) (2021)
- [ProDiff: Progressive Fast Diffusion Model For High-Quality Text-to-Speech](https://mlsanity.com/paper/2207.06389/) (2022)
- [Guided-TTS: A Diffusion Model for Text-to-Speech via Classifier Guidance](https://mlsanity.com/paper/2111.11755/) (2021)
- [Diff-HierVC: Diffusion-based Hierarchical Voice Conversion with Robust Pitch Generation and Masked Prior for Zero-shot Speaker Adaptation](https://mlsanity.com/paper/2311.04693/) (2023)
- [Digital Voicing of Silent Speech](https://mlsanity.com/paper/2010.02960/) (2020)
- [Diff-E: Diffusion-based Learning for Decoding Imagined Speech EEG](https://mlsanity.com/paper/2307.14389/) (2023)
- [An Improved Model for Voicing Silent Speech](https://mlsanity.com/paper/2106.01933/) (2021)
- [Fast Yet Effective Speech Emotion Recognition with Self-distillation](https://mlsanity.com/paper/2210.14636/) (2022)

## Built on by (most cited first, 4 of 4)

- [TinyMyo: a Tiny Foundation Model for Flexible EMG Signal Processing at the Edge](https://mlsanity.com/paper/2512.15729/) (2025)
- [Articulatory Feature Prediction from Surface EMG during Speech Production](https://mlsanity.com/paper/2505.13814/) (2025)
- [An Approach to Simultaneous Acquisition of Real-Time MRI Video, EEG, and Surface EMG for Articulatory, Brain, and Muscle Activity During Speech Production](https://mlsanity.com/paper/2603.04840/) (2026)
- [Cross-Modal Masking for Robust Silent Speech Synthesis Using sEMG and Lipreading](https://mlsanity.com/paper/2606.09667/) (2026)

## Abstract (arXiv)

Electromyography-to-Speech (ETS) conversion has demonstrated its potential for silent speech interfaces by generating audible speech from Electromyography (EMG) signals during silent articulations. ETS models usually consist of an EMG encoder which converts EMG signals to acoustic speech features, and a vocoder which then synthesises the speech signals. Due to an inadequate amount of available data and noisy signals, the synthesised speech often exhibits a low level of naturalness. In this work, we propose Diff-ETS, an ETS model which uses a score-based diffusion probabilistic model to enhance the naturalness of synthesised speech. The diffusion model is applied to improve the quality of the acoustic features predicted by an EMG encoder. In our experiments, we evaluated fine-tuning the diffusion model on predictions of a pre-trained EMG encoder, and training both models in an end-to-end fashion. We compared Diff-ETS with a baseline ETS model without diffusion using objective metrics and a listening test. The results indicated the proposed Diff-ETS significantly improved speech naturalness over the baseline.