mlsanityData to 7 Oct 2026

HiFTNet: A Fast High-Quality Neural Vocoder with Harmonic-plus-Noise Filter and Inverse Short Time Fourier Transform

arXiv 2309.09493 ↗First posted 18 Sep 2023eess.AS cs.AI cs.SDcited by 10

Not on the map. Why?

Abstract (arXiv)

Recent advancements in speech synthesis have leveraged GAN-based networks like HiFi-GAN and BigVGAN to produce high-fidelity waveforms from mel-spectrograms. However, these networks are computationally expensive and parameter-heavy. iSTFTNet addresses these limitations by integrating inverse short-time Fourier transform (iSTFT) into the network, achieving both speed and parameter efficiency. In this paper, we introduce an extension to iSTFTNet, termed HiFTNet, which incorporates a harmonic-plus-noise source filter in the time-frequency domain that uses a sinusoidal source from the fundamental frequency (F0) inferred via a pre-trained F0 estimation network for fast inference speed. Subjective evaluations on LJSpeech show that our model significantly outperforms both iSTFTNet and HiFi-GAN, achieving ground-truth-level performance. HiFTNet also outperforms BigVGAN-base on LibriTTS for unseen speakers and achieves comparable performance to BigVGAN while being four times faster with only 1/6 of the parameters. Our work sets a new benchmark for efficient, high-quality neural vocoding, paving the way for real-time applications that demand high quality speech synthesis.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 26 of its 33 references that are in the map. First posted on arXiv 18 Sep 2023.

Builds on

26 of 33 references.

References · 26

  1. 2017Decoupled Weight Decay Regularizationcited by 1,722
  2. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
  3. 2016Least Squares Generative Adversarial Networkscited by 1,000
  4. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 800
  5. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 710
  6. 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 639
  7. 2019MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesiscited by 351
  8. 2019Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogramcited by 337
  9. 2022BigVGAN: A Universal Neural Vocoder with Large-Scale Trainingcited by 245
  10. 2019AUTOVC: Zero-Shot Voice Style Transfer with Only Autoencoder Losscited by 189
  11. 2021DiffSinger: Singing Voice Synthesis via Shallow Diffusion Mechanismcited by 154
  12. 2020FastPitch: Parallel Text-to-speech with Pitch Predictioncited by 150
14 more
  1. 2023StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Modelscited by 148
  2. 2020Neural Networks Fail to Learn Periodic Functions and How to Fix Itcited by 97
  3. 2021UnivNet: A Neural Vocoder with Multi-Resolution Spectrogram Discriminators for High-Fidelity Waveform Generationcited by 71
  4. 2021StarGANv2-VC: A Diverse, Unsupervised, Non-parallel Framework for Natural-Sounding Voice Conversioncited by 57
  5. 2019Neural source-filter waveform models for statistical parametric speech synthesiscited by 46
  6. 2020HiFiSinger: Towards High-Fidelity Neural Singing Voice Synthesiscited by 39
  7. 2022NANSY++: Unified Voice Synthesis with Neural Analysis and Synthesiscited by 38
  8. 2022StyleTTS: A Style-Based Generative Model for Natural and Diverse Text-to-Speech Synthesiscited by 32
  9. 2022iSTFTNet: Fast and Lightweight Mel-Spectrogram Vocoder Incorporating Inverse Short-Time Fourier Transformcited by 30
  10. 2020CycleGAN-VC3: Examining and Improving CycleGAN-VCs for Mel-spectrogram Conversioncited by 25
  11. 2022Zero-Shot Voice Conditioning for Denoising Diffusion TTS Modelscited by 20
  12. 2022StyleTTS-VC: One-Shot Voice Conversion by Knowledge Transfer from Style-Based TTS Modelscited by 9
  13. 2022Unified Source-Filter GAN with Harmonic-plus-Noise Source Excitation Generationcited by 5
  14. 2023SLMGAN: Exploiting Speech Language Model Representations for Unsupervised Zero-Shot Voice Conversion in GANscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 1; 2026: 6.

10 papers.

  1. 2024FunAudioLLM: Voice Understanding and Generation Foundation Models for Natural Interaction Between Humans and LLMscited by 118
  2. 2024Recent Advances in Speech Language Models: A Surveycited by 85
  3. 2024StyleTTS-ZS: Efficient High-Quality Zero-Shot Text-to-Speech Synthesis with Distilled Time-Varying Style Diffusioncited by 6
  4. 2025MELA-TTS: Joint transformer-diffusion model with representation alignment for speech synthesiscited by 2
  5. 2026CosyAccent: Duration-Controllable Accent Normalization Using Source-Synthesis Training Datacited by 2
  6. 2026Controllable Accent Normalization via Discrete Diffusioncited by 0
  7. 2026Cross-Modal Masking for Robust Silent Speech Synthesis Using sEMG and Lipreadingcited by 0
  8. 2026Enhancing Flow Matching with A Unified Guidance Framework for Efficient and Robust Speech Synthesiscited by 0
  9. 2026TokAN: Accent Normalization Using Self-Supervised Speech Tokenscited by 0
  10. 2026Poly-InstructTTS: Learning In-the-Wild Expressive Speech Synthesis from Open-Ended Instructionscited by 0