mlsanityData to 7 Oct 2026

Vector Quantized Diffusion Model Based Speech Bandwidth Extension

arXiv 2409.05784 ↗First posted 9 Sep 2024cs.SD eess.AScited by 4

On the map: Diffusion models → Audio, speech and music

Abstract (arXiv)

Recent advancements in neural audio codec (NAC) unlock new potential in audio signal processing. Studies have increasingly explored leveraging the latent features of NAC for various speech signal processing tasks. This paper introduces the first approach to speech bandwidth extension (BWE) that utilizes the discrete features obtained from NAC. By restoring high-frequency details within highly compressed discrete tokens, this approach enhances speech intelligibility and naturalness. Based on Vector Quantized Diffusion, the proposed framework combines the strengths of advanced NAC, diffusion models, and Mamba-2 to reconstruct high-frequency speech components. Extensive experiments demonstrate that this method exhibits superior performance across both log-spectral distance and ViSQOL, significantly improving speech quality.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 25 of its 29 references that are in the map. First posted on arXiv 9 Sep 2024.

Builds on

25 of 29 references.

Closest ancestors · 1

Same direction, newest first

  1. 2021NU-Wave: A Diffusion Probabilistic Model for Neural Audio Upsampling6 Apr

Other references · 24

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  5. 2016Layer Normalizationcited by 5,579
  6. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  7. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,424
  8. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  9. 2024Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Dualitycited by 1,283
  10. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
  11. 2021Vector Quantized Diffusion Model for Text-to-Image Synthesiscited by 773
  12. 2022High Fidelity Neural Audio Compressioncited by 673
12 more
  1. 2023High-Fidelity Audio Compression with Improved RVQGANcited by 423
  2. 2023HiFi-Codec: Group-residual Vector quantization for High Fidelity Audio Codeccited by 95
  3. 2022Improved Vector Quantized Diffusion Modelscited by 62
  4. 2024Mamba in Speech: Towards an Alternative to Self-Attentioncited by 58
  5. 2023AudioSR: Versatile Audio Super-resolution at Scalecited by 39
  6. 2022Neural Vocoder is All You Need for Speech Super-resolutioncited by 37
  7. 2022VoiceFixer: A Unified Framework for High-Fidelity Speech Restorationcited by 37
  8. 2022NU-Wave 2: A General Neural Audio Upsampling Model for Various Sampling Ratescited by 36
  9. 2024Speech Slytherin: Examining the Performance and Efficiency of Mamba for Speech Separation, Recognition, and Synthesiscited by 22
  10. 2024Exploring the Capability of Mamba in Speech Applicationscited by 19
  11. 2022AERO: Audio Super Resolution in the Spectral Domaincited by 17
  12. 2023BAE-Net: A Low complexity and high fidelity Bandwidth-Adaptive neural network for speech super-resolutioncited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 2.

4 papers.

  1. 2025Discrete Diffusion for Generative Modeling of Text-Aligned Speech Tokenscited by 7
  2. 2025Learning to Upsample and Upmix Audio in the Latent Domaincited by 5
  3. 2026CodecFlow: Efficient Bandwidth Extension via Conditional Flow Matching in Neural Codec Latent Spacecited by 1
  4. 2026P2Flow: Phoneme-aware Progressive Flow Matching for Extreme Speech Super-Resolutioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Absorbing Discrete Diffusion for Speech Enhancement
  2. 2023Generative De-Quantization for Neural Speech Codec via Latent Diffusion
  3. 2025On the Design of Diffusion-based Neural Speech Codecs
  4. 2026DiffVQE2: An Efficient Low-delay Diffusion Model for Acoustic Echo and Noise Control
  5. 2026Rethinking Language Model-Based Generative Speech Enhancement in the Latent Space of a Neural Audio Codec
  6. 2023Diffusion-based Signal Refiner for Speech Enhancement and Separation
  7. 2022Analysing Diffusion-based Generative Approaches versus Discriminative Approaches for Speech Restoration
  8. 2023A Survey on Audio Diffusion Models: Text To Speech Synthesis and Enhancement in Generative AI