mlsanityData to 14 Oct 2026

Low Bit-Rate Speech Coding with VQ-VAE and a WaveNet Decoder

arXiv 1910.06464 ↗First posted 14 Oct 2019cs.LG cs.SD eess.AScited by 49

Not on the map. Why?

Abstract (arXiv)

In order to efficiently transmit and store speech signals, speech codecs create a minimally redundant representation of the input signal which is then decoded at the receiver with the best possible perceptual quality. In this work we demonstrate that a neural network architecture based on VQ-VAE with a WaveNet decoder can be used to perform very low bit-rate speech coding with high reconstruction quality. A prosody-transparent and speaker-independent model trained on the LibriSpeech corpus coding audio at 1.6 kbps exhibits perceptual quality which is around halfway between the MELP codec at 2.4 kbps and AMR-WB codec at 23.05 kbps. In addition, when training on high-quality recorded speech with the test speaker included in the training set, a model coding speech at 1.6 kbps produces output of similar perceptual quality to that generated by AMR-WB at 23.05 kbps.

Key facts. Cited by 49 papers in the map (data through 14 Oct 2026). First posted on arXiv 14 Oct 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 10; 2022: 13; 2023: 9; 2024: 10; 2025: 6; 2026: 1.

49 papers.

Sort:

By area:

  1. 2021SoundStream: An End-to-End Neural Audio Codeccited by 771
  2. 2022High Fidelity Neural Audio Compressioncited by 687
  3. 2023High-Fidelity Audio Compression with Improved RVQGANcited by 431
  4. 2021Speech Resynthesis from Discrete Disentangled Self-Supervised Representationscited by 243
  5. 2023VampNet: Music Generation via Masked Acoustic Token Modelingcited by 57
  6. 2024SemantiCodec: An Ultra Low Bitrate Semantic Audio Codec for General Soundcited by 45
  7. 2024SNAC: Multi-Scale Neural Audio Codeccited by 43
  8. 2023JEN-1: Text-Guided Universal Music Generation with Omnidirectional Diffusion Modelscited by 41
  9. 2023RepCodec: A Speech Representation Codec for Speech Tokenizationcited by 27
  10. 2022Audio-Visual Speech Codecs: Rethinking Audio-Visual Speech Enhancement by Re-Synthesiscited by 24
  11. 2023JEN-1 Composer: A Unified Framework for High-Fidelity Multi-Track Music Generationcited by 17
  12. 2023EDMSound: Spectrogram Based Diffusion Models for Efficient and High-Quality Audio Synthesiscited by 15
37 more
  1. 2021Neural Distributed Source Codingcited by 13
  2. 2024Low Frame-rate Speech Codec: a Codec Designed for Fast High-quality Speech LLM Training and Inferencecited by 11
  3. 2022Disentangled Feature Learning for Real-Time Neural Speech Codingcited by 10
  4. 2022A Multi-Stage Multi-Codebook VQ-VAE Approach to High-Performance Neural TTScited by 9
  5. 2023Generative De-Quantization for Neural Speech Codec via Latent Diffusioncited by 9
  6. 2021Enhancing into the codec: Noise Robust Speech Coding with Vector-Quantized Autoencoderscited by 7
  7. 2021HARP-Net: Hyper-Autoencoded Reconstruction Propagation for Scalable Neural Audio Codingcited by 7
  8. 2022End-to-End Neural Speech Coding for Real-Time Communicationscited by 7
  9. 2022Latent-Domain Predictive Neural Speech Codingcited by 7
  10. 2024Takin: A Cohort of Superior Quality Zero-shot Speech Generation Modelscited by 7
  11. 2021Evidential Softmax for Sparse Multimodal Distributions in Deep Generative Modelscited by 6
  12. 2022Disentangling speech from surroundings with neural embeddingscited by 6
  13. 2024Learning Source Disentanglement in Neural Audio Codeccited by 6
  14. 2021Scalable and Efficient Neural Speech Coding: A Hybrid Designcited by 5
  15. 2022Ultra-Low-Bitrate Speech Coding with Pretrained Transformerscited by 5
  16. 2024Gull: A Generative Multifunctional Audio Codeccited by 5
  17. 2025SecoustiCodec: Cross-Modal Aligned Streaming Single-Codecbook Speech Codeccited by 5
  18. 2022End-to-End Binaural Speech Synthesiscited by 4
  19. 2022Cross-Scale Vector Quantization for Scalable Neural Speech Codingcited by 3
  20. 2022NESC: Robust Neural End-2-End Speech Coding with GANscited by 3
  21. 2024PSCodec: A Series of High-Fidelity Low-bitrate Neural Speech Codecs Leveraging Prompt Encoderscited by 3
  22. 2025Universal Speech Token Learning via Low-Bitrate Neural Codec and Pretrained Representationscited by 3
  23. 2025Versatile Physics-based Character Control with Hybrid Latent Representationcited by 3
  24. 2021Cognitive Coding of Speechcited by 2
  25. 2022Deep Reinforcement Learning with Vector Quantized Encodingcited by 2
  26. 2024Neural Speech and Audio Coding: Modern AI Technology Meets Traditional Codecscited by 2
  27. 2022Conditional variational autoencoder to improve neural audio synthesis for polyphonic music soundcited by 1
  28. 2023AudioFormer: Audio Transformer learns audio feature representations from discrete acoustic codescited by 1
  29. 2023QS-TTS: Towards Semi-Supervised Text-to-Speech Synthesis via Vector-Quantized Self-Supervised Speech Representation Learningcited by 1
  30. 2024Spiking Music: Audio Compression with Event Based Auto-encoderscited by 1
  31. 2024Enhancing Interpretability in Deep Reinforcement Learning through Semantic Clusteringcited by 1
  32. 2025Universal Semantic Disentangled Privacy-preserving Speech Representation Learningcited by 1
  33. 2021Low Bit-Rate Wideband Speech Coding: A Deep Generative Model based Approachcited by 0
  34. 2021Unified Signal Compression Using a GAN with Iterative Latent Representation Optimizationcited by 0
  35. 2025Make Some Noise: Towards LLM audio reasoning and generation using sound tokenscited by 0
  36. 2025Normality Prior Guided Multi-Semantic Fusion Network for Unsupervised Image Anomaly Detectioncited by 0
  37. 2026Train to Stitch: Topology-Preserving Regularization for Interoperable Representationscited by 0