mlsanityData to 19 Oct 2026

Comparative Analysis of Fast and High-Fidelity Neural Vocoders for Low-Latency Streaming Synthesis in Resource-Constrained Environments

arXiv 2506.03554 ↗First posted 4 Jun 2025cs.SD eess.AScited by 0

On the map: Speech and audio language models → Vocoders and speech synthesis

Abstract (arXiv)

In real-time speech synthesis, neural vocoders often require low-latency synthesis through causal processing and streaming. However, streaming introduces inefficiencies absent in batch synthesis, such as limited parallelism, inter-frame dependency management, and parameter loading overhead. This paper proposes multi-stream Wavehax (MS-Wavehax), an efficient neural vocoder for low-latency streaming, by extending the aliasing-free neural vocoder Wavehax with multi-stream decomposition. We analyze the latency-throughput trade-off in a CPU-only environment and identify key bottlenecks in streaming neural vocoders. Our findings provide practical insights for optimizing chunk sizes and designing vocoders tailored to specific application demands and hardware constraints. Furthermore, our subjective evaluations show that MS-Wavehax delivers high speech quality under causal and non-causal conditions while being remarkably compact and easily deployable in resource-constrained environments.

Key facts. No citing papers in the map yet (data through 19 Oct 2026). Placed in Vocoders and speech synthesis, one of 11 directions in Speech and audio language models. Builds on 29 of its 40 references that are in the map. First posted on arXiv 4 Jun 2025.

Builds on

29 of 40 references.

Closest ancestors · 6

Same direction, newest first

  1. 2023Matcha-TTS: A fast TTS architecture with conditional flow matching6 Sep
  2. 2023iSTFTNet2: Faster and More Lightweight iSTFT-Based Neural Vocoder Using 1D-2D CNN14 Aug
  3. 2022Lightweight and High-Fidelity End-to-End Text-to-Speech with Multi-Band Generation and Inverse Short-Time Fourier Transform28 Oct
  4. 2022iSTFTNet: Fast and Lightweight Mel-Spectrogram Vocoder Incorporating Inverse Short-Time Fourier Transform4 Mar
  5. 2021High Quality Streaming Speech Synthesis with Low, Sentence-Length-Independent Latency17 Nov
  6. 2021AC-VC: Non-parallel Low Latency Phonetic Posteriorgrams Based Voice Conversion12 Nov

Other references · 23

  1. 2017Attention Is All You Needcited by 61,180
  2. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,093
  3. 2022Flow Matching for Generative Modelingcited by 5,905
  4. 2016Layer Normalizationcited by 5,866
  5. 2022A ConvNet for the 2020scited by 3,617
  6. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,462
  7. 2022UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022cited by 552
  8. 2019MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesiscited by 436
  9. 2022BigVGAN: A Universal Neural Vocoder with Large-Scale Trainingcited by 315
  10. 2023Vocos: Closing the gap between time-domain and Fourier-based neural vocoders for high-quality audio synthesiscited by 218
  11. 2020Developing Real-time Streaming Transformer Transducer for Speech Recognition on Large-scale Datasetcited by 120
  12. 2021UnivNet: A Neural Vocoder with Multi-Resolution Spectrogram Discriminators for High-Fidelity Waveform Generationcited by 107
11 more
  1. 2020Multi-band MelGAN: Faster Waveform Generation for High-Quality Text-to-Speechcited by 91
  2. 2017JSUT corpus: free large-scale Japanese speech corpus for end-to-end speech synthesiscited by 80
  3. 2019JVS corpus: free Japanese multi-speaker voice corpuscited by 36
  4. 2021Fre-GAN: Adversarial Frequency-consistent Audio Synthesiscited by 30
  5. 2019Incremental Text-to-Speech Synthesis with Prefix-to-Prefix Frameworkcited by 19
  6. 2024FreGrad: Lightweight and Fast Frequency-aware Diffusion Vocodercited by 12
  7. 2020What the Future Brings: Investigating the Impact of Lookahead for Incremental Neural TTScited by 10
  8. 2023APNet: An All-Frame-Level Neural Vocoder Incorporating Direct Prediction of Amplitude and Phase Spectracited by 10
  9. 2022Streaming non-autoregressive model for any-to-many voice conversioncited by 4
  10. 2021Low-Latency Real-Time Non-Parallel Voice Conversion based on Cyclic Variational Autoencoder and Multiband WaveRNN with Data-Driven Linear Predictioncited by 3
  11. 2024Wavehax: Aliasing-Free Neural Waveform Synthesis Based on 2D Convolution and Harmonic Prior for Reliable Complex Spectrogram Estimationcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026An Ultra-Low Latency, End-to-End Streaming Speech Synthesis Architecture via Block-Wise Generation and Depth-Wise Codec Decoding
  2. 2021VocBench: A Neural Vocoder Benchmark for Speech Synthesis
  3. 2021Universal Neural Vocoding with Parallel WaveNet
  4. 2024StreamVC: Real-Time Low-Latency Voice Conversion
  5. 2024Ultra-lightweight Neural Differential DSP Vocoder For High Quality Speech Synthesis
  6. 2021FlowVocoder: A small Footprint Neural Vocoder based Normalizing flow for Speech Synthesis
  7. 2025WaveFM: A High-Fidelity and Efficient Vocoder Based on Flow Matching
  8. 2026ZipCodec: Ultra-Low-Frame-Rate Streaming Speech Coding