# Comparative Analysis of Fast and High-Fidelity Neural Vocoders for Low-Latency Streaming Synthesis in Resource-Constrained Environments

arXiv: https://arxiv.org/abs/2506.03554 · HTML: https://mlsanity.com/paper/2506.03554/

## Key facts

- No citing papers in the map yet (data through 19 Oct 2026).
- Placed in Vocoders and speech synthesis, one of 11 directions in Speech and audio language models.
- Builds on 29 of its 40 references that are in the map.
- First posted on arXiv 4 Jun 2025.

## Placed in

- Speech and audio language models / [Vocoders and speech synthesis](https://mlsanity.com/speech/vocoders-and-speech-synthesis/)

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis](https://mlsanity.com/paper/2010.05646/) (2020)
- [UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022](https://mlsanity.com/paper/2204.02152/) (2022)
- [MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesis](https://mlsanity.com/paper/1910.06711/) (2019)
- [BigVGAN: A Universal Neural Vocoder with Large-Scale Training](https://mlsanity.com/paper/2206.04658/) (2022)
- [Vocos: Closing the gap between time-domain and Fourier-based neural vocoders for high-quality audio synthesis](https://mlsanity.com/paper/2306.00814/) (2023)
- [Matcha-TTS: A fast TTS architecture with conditional flow matching](https://mlsanity.com/paper/2309.03199/) (2023)
- [Developing Real-time Streaming Transformer Transducer for Speech Recognition on Large-scale Dataset](https://mlsanity.com/paper/2010.11395/) (2020)
- [UnivNet: A Neural Vocoder with Multi-Resolution Spectrogram Discriminators for High-Fidelity Waveform Generation](https://mlsanity.com/paper/2106.07889/) (2021)
- [Multi-band MelGAN: Faster Waveform Generation for High-Quality Text-to-Speech](https://mlsanity.com/paper/2005.05106/) (2020)
- [JSUT corpus: free large-scale Japanese speech corpus for end-to-end speech synthesis](https://mlsanity.com/paper/1711.00354/) (2017)
- [iSTFTNet: Fast and Lightweight Mel-Spectrogram Vocoder Incorporating Inverse Short-Time Fourier Transform](https://mlsanity.com/paper/2203.02395/) (2022)
- [JVS corpus: free Japanese multi-speaker voice corpus](https://mlsanity.com/paper/1908.06248/) (2019)
- [Fre-GAN: Adversarial Frequency-consistent Audio Synthesis](https://mlsanity.com/paper/2106.02297/) (2021)
- [High Quality Streaming Speech Synthesis with Low, Sentence-Length-Independent Latency](https://mlsanity.com/paper/2111.09052/) (2021)
- [Incremental Text-to-Speech Synthesis with Prefix-to-Prefix Framework](https://mlsanity.com/paper/1911.02750/) (2019)
- [Lightweight and High-Fidelity End-to-End Text-to-Speech with Multi-Band Generation and Inverse Short-Time Fourier Transform](https://mlsanity.com/paper/2210.15975/) (2022)
- [FreGrad: Lightweight and Fast Frequency-aware Diffusion Vocoder](https://mlsanity.com/paper/2401.10032/) (2024)
- [What the Future Brings: Investigating the Impact of Lookahead for Incremental Neural TTS](https://mlsanity.com/paper/2009.02035/) (2020)
- [APNet: An All-Frame-Level Neural Vocoder Incorporating Direct Prediction of Amplitude and Phase Spectra](https://mlsanity.com/paper/2305.07952/) (2023)
- [iSTFTNet2: Faster and More Lightweight iSTFT-Based Neural Vocoder Using 1D-2D CNN](https://mlsanity.com/paper/2308.07117/) (2023)
- [AC-VC: Non-parallel Low Latency Phonetic Posteriorgrams Based Voice Conversion](https://mlsanity.com/paper/2111.06601/) (2021)
- [Streaming non-autoregressive model for any-to-many voice conversion](https://mlsanity.com/paper/2206.07288/) (2022)
- [Low-Latency Real-Time Non-Parallel Voice Conversion based on Cyclic Variational Autoencoder and Multiband WaveRNN with Data-Driven Linear Prediction](https://mlsanity.com/paper/2105.09858/) (2021)
- [Wavehax: Aliasing-Free Neural Waveform Synthesis Based on 2D Convolution and Harmonic Prior for Reliable Complex Spectrogram Estimation](https://mlsanity.com/paper/2411.06807/) (2024)

## Abstract (arXiv)

In real-time speech synthesis, neural vocoders often require low-latency synthesis through causal processing and streaming. However, streaming introduces inefficiencies absent in batch synthesis, such as limited parallelism, inter-frame dependency management, and parameter loading overhead. This paper proposes multi-stream Wavehax (MS-Wavehax), an efficient neural vocoder for low-latency streaming, by extending the aliasing-free neural vocoder Wavehax with multi-stream decomposition. We analyze the latency-throughput trade-off in a CPU-only environment and identify key bottlenecks in streaming neural vocoders. Our findings provide practical insights for optimizing chunk sizes and designing vocoders tailored to specific application demands and hardware constraints. Furthermore, our subjective evaluations show that MS-Wavehax delivers high speech quality under causal and non-causal conditions while being remarkably compact and easily deployable in resource-constrained environments.