VoCodec: A Low-bitrate Streamable Neural Speech Codec with Voicing-driven Quantization
On the map: Speech and audio language models → Vocoders and speech synthesis
Abstract (arXiv)
Neural speech codecs are key to speech transmission and storage, but most use uniform quantization across frames, allocating the same bitrate regardless of content and wasting bits. We propose VoCodec, a low-bitrate streamable neural speech codec with voicing-driven quantization that assigns higher bitrate to voiced frames and lower bitrate to unvoiced frames according to perceptual sensitivity. VoCodec embeds a voicing detector in a fully causal encoder-quantizer-decoder neural coding framework, using residual scalar-vector quantization for voiced frames and simple scalar quantization for unvoiced ones. Experiments show that on the LibriTTS dataset at a 16 kHz sampling rate, VoCodec outperforms baseline neural speech codecs even at a bitrate as low as 1.1 kbps. Our further experiments also confirm that introducing voicing-driven quantization can effectively reduce the bitrate by approximately 27% compared with uniform quantization strategy.
Key facts. No citing papers in the map yet (data through 19 Oct 2026). Placed in Vocoders and speech synthesis, one of 11 directions in Speech and audio language models. Builds on 17 of its 30 references that are in the map. First posted on arXiv 4 Jun 2026.
Builds on
17 of 30 references.
Closest ancestors · 8
Same direction, newest first
- 2024MDCTCodec: A Lightweight MDCT-based Neural Audio Codec towards High Sampling Rate and Low Bitrate Scenarios1 Nov
- 2024ERVQ: Enhanced Residual Vector Quantization with Intra-and-Inter-Codebook Optimization for Neural Audio Codecs16 Oct
- 2024BigCodec: Pushing the Limits of Low-Bitrate Neural Speech Codec9 Sep
- 2023Matcha-TTS: A fast TTS architecture with conditional flow matching6 Sep
- 2023AudioDec: An Open-source Streaming High-fidelity Neural Audio Codec26 May
- 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers5 Jan
- 2021SoundStream: An End-to-End Neural Audio Codec7 Jul
- 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech11 Jun
Other references · 9
- 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,462
- 2022High Fidelity Neural Audio Compressioncited by 917
- 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 864
- 2022AudioLM: a Language Modeling Approach to Audio Generationcited by 736
- 2023High-Fidelity Audio Compression with Improved RVQGANcited by 619
- 2023Finite Scalar Quantization: VQ-VAE Made Simplecited by 498
- 2017SEGAN: Speech Enhancement Generative Adversarial Networkcited by 262
- 2020ViSQOL v3: An Open Source Production Ready Objective Speech and Audio Metriccited by 120
- 2024SemantiCodec: An Ultra Low Bitrate Semantic Audio Codec for General Soundcited by 84
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026An Ultra-Low-Bitrate Neural Speech Codec with Plain-to-Pseudo Synergistic Vector Quantization
- 2026VoCodec: An Efficient Lightweight Low-Bitrate Speech Codec
- 2026ZipCodec: Ultra-Low-Frame-Rate Streaming Speech Coding
- 2024Neural Speech Coding for Real-time Communications using Constant Bitrate Scalar Quantization
- 2024ESC: Efficient Speech Coding with Cross-Scale Residual Vector Quantized Transformers
- 2024Low Frame-rate Speech Codec: a Codec Designed for Fast High-quality Speech LLM Training and Inference
- 2023LMCodec: A Low Bitrate Speech Codec With Causal Transformer Models
- 2022Ultra-Low-Bitrate Speech Coding with Pretrained Transformers