# Autovocoder: Fast Waveform Generation from a Learned Speech Representation using Differentiable Digital Signal Processing

arXiv: https://arxiv.org/abs/2211.06989 · HTML: https://mlsanity.com/paper/2211.06989/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Nov 2022.

## Built on by (most cited first, 5 of 5)

- [FastFit: Towards Real-Time Iterative Neural Vocoder by Replacing U-Net Encoder With Multiple STFTs](https://mlsanity.com/paper/2305.10823/) (2023)
- [Designing Neural Synthesizers for Low-Latency Interaction](https://mlsanity.com/paper/2503.11562/) (2025)
- [MiSTR: Multi-Modal iEEG-to-Speech Synthesis with Transformer-Based Prosody Prediction and Neural Phase Reconstruction](https://mlsanity.com/paper/2508.03166/) (2025)
- [Prosody-Guided Harmonic Attention for Phase-Coherent Neural Vocoding in the Complex Spectrum](https://mlsanity.com/paper/2601.14472/) (2026)
- [Brain-to-Speech: Prosody Feature Engineering and Transformer-Based Reconstruction](https://mlsanity.com/paper/2604.05751/) (2026)

## Abstract (arXiv)

Most state-of-the-art Text-to-Speech systems use the mel-spectrogram as an intermediate representation, to decompose the task into acoustic modelling and waveform generation. A mel-spectrogram is extracted from the waveform by a simple, fast DSP operation, but generating a high-quality waveform from a mel-spectrogram requires computationally expensive machine learning: a neural vocoder. Our proposed ``autovocoder'' reverses this arrangement. We use machine learning to obtain a representation that replaces the mel-spectrogram, and that can be inverted back to a waveform using simple, fast operations including a differentiable implementation of the inverse STFT. The autovocoder generates a waveform 5 times faster than the DSP-based Griffin-Lim algorithm, and 14 times faster than the neural vocoder HiFi-GAN. We provide perceptual listening test results to confirm that the speech is of comparable quality to HiFi-GAN in the copy synthesis task.