mlsanityData to 7 Oct 2026

Vocos: Closing the gap between time-domain and Fourier-based neural vocoders for high-quality audio synthesis

arXiv 2306.00814 ↗First posted 1 Jun 2023cs.SD cs.LG eess.AScited by 150

Not on the map. Why?

Cited by 47 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Recent advancements in neural vocoding are predominantly driven by Generative Adversarial Networks (GANs) operating in the time-domain. While effective, this approach neglects the inductive bias offered by time-frequency representations, resulting in reduntant and computionally-intensive upsampling operations. Fourier-based time-frequency representation is an appealing alternative, aligning more accurately with human auditory perception, and benefitting from well-established fast algorithms for its computation. Nevertheless, direct reconstruction of complex-valued spectrograms has been historically problematic, primarily due to phase recovery issues. This study seeks to close this gap by presenting Vocos, a new model that directly generates Fourier spectral coefficients. Vocos not only matches the state-of-the-art in audio quality, as demonstrated in our evaluations, but it also substantially improves computational efficiency, achieving an order of magnitude increase in speed compared to prevailing time-domain neural vocoding approaches. The source code and model weights have been open-sourced at https://github.com/gemelo-ai/vocos.

Key facts. Cited by 150 papers in the map (data through 7 Oct 2026). Builds on 38 of its 61 references that are in the map. First posted on arXiv 1 Jun 2023.

Builds on

38 of 61 references.

References · 38

  1. 2017Attention Is All You Needcited by 59,932
  2. 2022A ConvNet for the 2020scited by 3,545
  3. 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 1,998
  4. 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
  5. 2023Mastering Diverse Domains through World Modelscited by 1,193
  6. 2021Alias-Free Generative Adversarial Networkscited by 1,181
  7. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
  8. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 800
  9. 2021SoundStream: An End-to-End Neural Audio Codeccited by 756
  10. 2022High Fidelity Neural Audio Compressioncited by 673
  11. 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 639
  12. 2022AudioLM: a Language Modeling Approach to Audio Generationcited by 611
26 more
  1. 2022UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022cited by 381
  2. 2019MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesiscited by 351
  3. 2018WaveGlow: A Flow-based Generative Network for Speech Synthesiscited by 308
  4. 2018cGANs with Projection Discriminatorcited by 253
  5. 2022BigVGAN: A Universal Neural Vocoder with Large-Scale Trainingcited by 245
  6. 2021Speech Resynthesis from Discrete Disentangled Self-Supervised Representationscited by 243
  7. 2018Adversarial Audio Synthesiscited by 222
  8. 2018Efficient Neural Audio Synthesiscited by 218
  9. 2017Parallel WaveNet: Fast High-Fidelity Speech Synthesiscited by 214
  10. 2019GANSynth: Adversarial Neural Audio Synthesiscited by 114
  11. 2016SampleRNN: An Unconditional End-to-End Neural Audio Generation Modelcited by 110
  12. 2018LPCNet: Improving Neural Speech Synthesis Through Linear Predictioncited by 109
  13. 2020Neural Networks Fail to Learn Periodic Functions and How to Fix Itcited by 97
  14. 2020ViSQOL v3: An Open Source Production Ready Objective Speech and Audio Metriccited by 76
  15. 2021RAVE: A variational autoencoder for fast and high-quality neural audio synthesiscited by 72
  16. 2021UnivNet: A Neural Vocoder with Multi-Resolution Spectrogram Discriminators for High-Fidelity Waveform Generationcited by 71
  17. 2022ICASSP 2022 Deep Noise Suppression Challengecited by 57
  18. 2020A Spectral Energy Distance for Parallel Speech Synthesiscited by 42
  19. 2021Chunked Autoregressive GAN for Conditional Waveform Synthesiscited by 38
  20. 2022iSTFTNet: Fast and Lightweight Mel-Spectrogram Vocoder Incorporating Inverse Short-Time Fourier Transformcited by 30
  21. 2022Musika! Fast Infinite Waveform Music Generationcited by 24
  22. 2022Avocodo: Generative Adversarial Network for Artifact-free Vocodercited by 18
  23. 2022WavThruVec: Latent speech representation as intermediate features for neural speech synthesiscited by 17
  24. 2021GAN Vocoder: Multi-Resolution Discriminator Is All You Needcited by 16
  25. 2018Generative adversarial network-based approach to signal reconstruction from magnitude spectrogramscited by 8
  26. 2018Phase reconstruction from amplitude spectrograms based on von-Mises-distribution deep neural networkcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 23; 2025: 65; 2026: 61.

150 papers.

Sort:

By area:

  1. 2024F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matchingcited by 353
  2. 2024CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Modelscited by 282
  3. 2024MaskGCT: Zero-Shot Text-to-Speech with Masked Generative Codec Transformercited by 159
  4. 2024WavTokenizer: an Efficient Acoustic Discrete Codec Tokenizer for Audio Language Modelingcited by 124
  5. 2024VALL-E 2: Neural Codec Language Models are Human Parity Zero-Shot Text to Speech Synthesizerscited by 119
  6. 2024Addressing Representation Collapse in Vector Quantized Models with One Linear Layercited by 77
  7. 2024Autoregressive Speech Synthesis without Vector Quantizationcited by 65
  8. 2023SpeechX: Neural Codec Language Model as a Versatile Speech Transformercited by 64
  9. 2025MiMo-Audio: Audio Language Models are Few-Shot Learnerscited by 58
  10. 2025Recent Advances in Discrete Speech Tokens: A Reviewcited by 42
  11. 2024VALL-E R: Robust and Efficient Zero-Shot Text-to-Speech Synthesis via Monotonic Alignmentcited by 37
  12. 2025UniAVGen: Unified Audio and Video Generation with Asymmetric Cross-Modal Interactionscited by 35
38 more
  1. 2025ZipVoice: Fast and High-Quality Zero-Shot Text-to-Speech with Flow Matchingcited by 30
  2. 2024Towards Controllable Speech Synthesis in the Era of Large Language Models: A Systematic Surveycited by 22
  3. 2025XY-Tokenizer: Mitigating the Semantic-Acoustic Conflict in Low-Bitrate Speech Codecscited by 21
  4. 2025FocalCodec: Low-Bitrate Speech Coding via Focal Modulation Networkscited by 18
  5. 2024ControlSpeech: Towards Simultaneous and Independent Zero-shot Speaker Cloning and Zero-shot Language Style Controlcited by 17
  6. 2025FlowSE: Efficient and High-Quality Speech Enhancement via Flow Matchingcited by 17
  7. 2025Ming-UniAudio: Speech LLM for Joint Understanding, Generation and Editing with Unified Representationcited by 15
  8. 2024PeriodWave: Multi-Period Flow Matching for High-Fidelity Waveform Generationcited by 12
  9. 2025TaDiCodec: Text-aware Diffusion Speech Tokenizer for Speech Language Modelingcited by 12
  10. 2025CineBrain: A Large-Scale Multi-Modal Brain Dataset During Naturalistic Audiovisual Narrative Processingcited by 11
  11. 2026[b] = [d] - [t] + [p]: Self-supervised Speech Models Discover Phonological Vector Arithmeticcited by 11
  12. 2024Accelerating Codec-based Speech Synthesis with Multi-Token Prediction and Speculative Decodingcited by 10
  13. 2025InspireMusic: Integrating Super Resolution and Large Language Model for High-Fidelity Long-Form Music Generationcited by 10
  14. 2025UALM: Unified Audio Language Model for Understanding, Generation and Reasoningcited by 10
  15. 2024RFWave: Multi-band Rectified Flow for Audio Waveform Reconstructioncited by 9
  16. 2024Estimating the Completeness of Discrete Speech Unitscited by 9
  17. 2025From Faces to Voices: Learning Hierarchical Representations for High-quality Video-to-Speechcited by 9
  18. 2025OmniTalker: One-shot Real-time Text-Driven Talking Audio-Video Generation With Multimodal Style Mimickingcited by 9
  19. 2025From Turn-Taking to Synchronous Dialogue: A Survey of Full-Duplex Spoken Language Modelscited by 9
  20. 2025SpeakStream: Streaming Text-to-Speech with Interleaved Datacited by 8
  21. 2025FocalCodec-Stream: Streaming Low-Bitrate Speech Coding via Causal Distillationcited by 8
  22. 2025Towards Interactive Intelligence for Digital Humanscited by 8
  23. 2024LSCodec: Low-Bitrate and Speaker-Decoupled Discrete Speech Codeccited by 7
  24. 2024Efficient Generative Modeling with Residual Vector Quantization-Based Tokenscited by 7
  25. 2025WaveFM: A High-Fidelity and Efficient Vocoder Based on Flow Matchingcited by 7
  26. 2025MagiCodec: Simple Masked Gaussian-Injected Codec for High-Fidelity Reconstruction and Generationcited by 7
  27. 2025ZipVoice-Dialog: Non-Autoregressive Spoken Dialogue Generation with Flow Matchingcited by 7
  28. 2025DMOSpeech 2: Reinforcement Learning for Duration Prediction in Metric-Optimized Speech Synthesiscited by 7
  29. 2025Multi-Metric Preference Alignment for Generative Speech Restorationcited by 7
  30. 2024Small-E: Small Language Model with Linear Attention for Efficient Speech Synthesiscited by 6
  31. 2024StyleTTS-ZS: Efficient High-Quality Zero-Shot Text-to-Speech Synthesis with Distilled Time-Varying Style Diffusioncited by 6
  32. 2025FlowTSE: Target Speaker Extraction with Flow Matchingcited by 6
  33. 2025M3-TTS: Multi-modal DiT Alignment & Mel-latent for Zero-shot High-fidelity Speech Synthesiscited by 6
  34. 2024Fake it to make it: Using synthetic data to remedy the data shortage in joint multimodal speech-and-gesture synthesiscited by 5
  35. 2025SafeSpeech: Robust and Universal Voice Protection Against Malicious Speech Synthesiscited by 5
  36. 2025Accelerating Flow-Matching-Based Text-to-Speech via Empirically Pruned Step Samplingcited by 5
  37. 2025Learning to Upsample and Upmix Audio in the Latent Domaincited by 5
  38. 2025Vevo2: A Unified and Controllable Framework for Speech and Singing Voice Generationcited by 5

50 most cited of 150 All 150 (JSON)