Cross-Scale Vector Quantization for Scalable Neural Speech Coding
Not on the map. Why?
Abstract (arXiv)
Bitrate scalability is a desirable feature for audio coding in real-time communications. Existing neural audio codecs usually enforce a specific bitrate during training, so different models need to be trained for each target bitrate, which increases the memory footprint at the sender and the receiver side and transcoding is often needed to support multiple receivers. In this paper, we introduce a cross-scale scalable vector quantization scheme (CSVQ), in which multi-scale features are encoded progressively with stepwise feature fusion and refinement. In this way, a coarse-level signal is reconstructed if only a portion of the bitstream is received, and progressively improves the quality as more bits are available. The proposed CSVQ scheme can be flexibly applied to any neural audio coding network with a mirrored auto-encoder structure to achieve bitrate scalability. Subjective results show that the proposed scheme outperforms the classical residual VQ (RVQ) with scalability. Moreover, the proposed CSVQ at 3 kbps outperforms Opus at 9 kbps and Lyra at 3kbps and it could provide a graceful quality boost with bitrate increase.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 22 of its 29 references that are in the map. First posted on arXiv 7 Jul 2022.
Builds on
22 of 29 references.
References · 22
- 2017Neural Discrete Representation Learningcited by 4,650
- 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
- 2016Least Squares Generative Adversarial Networkscited by 1,000
- 2021SoundStream: An End-to-End Neural Audio Codeccited by 756
- 2019vq-wav2vec: Self-Supervised Learning of Discrete Speech Representationscited by 335
- 2016SampleRNN: An Unconditional End-to-End Neural Audio Generation Modelcited by 110
- 2018LPCNet: Improving Neural Speech Synthesis Through Linear Predictioncited by 109
- 2019Low Bit-Rate Speech Coding with VQ-VAE and a WaveNet Decodercited by 47
- 2020A Spectral Energy Distance for Parallel Speech Synthesiscited by 42
- 2018Differentiable Consistency Constraints for Improved Deep Speech Enhancementcited by 35
- 2017Wavenet based low rate speech codingcited by 32
- 2020ICASSP 2021 Deep Noise Suppression Challengecited by 27
10 more
- 2017End-to-End Optimized Speech Coding with Deep Neural Networkscited by 21
- 2021Generative Speech Coding with Predictive Variance Regularizationcited by 18
- 2019Cascaded Cross-Module Residual Learning towards Lightweight End-to-End Speech Codingcited by 15
- 2018High-quality speech coding with SampleRNNcited by 14
- 2019Improving Opus Low Bit Rate Quality with Neural Speech Synthesiscited by 9
- 2020Source coding of audio signals with a generative modelcited by 7
- 2020Efficient And Scalable Neural Residual Waveform Coding With Collaborative Quantizationcited by 7
- 2021Enhancing into the codec: Noise Robust Speech Coding with Vector-Quantized Autoencoderscited by 7
- 2021HARP-Net: Hyper-Autoencoded Reconstruction Propagation for Scalable Neural Audio Codingcited by 7
- 2022End-to-End Neural Speech Coding for Real-Time Communicationscited by 7
Built on by
Citing papers per year: 2022: 1; 2023: 2.
3 papers.