mlsanityData to 19 Oct 2026

Entropy-Guided GRVQ for Ultra-Low Bitrate Neural Speech Codec

arXiv 2603.01476 ↗First posted 2 Mar 2026eess.AS eess.SPcited by 0

On the map: Speech and audio language models → Vocoders and speech synthesis

Abstract (arXiv)

Neural audio codec (NAC) is essential for reconstructing high-quality speech signals and generating discrete representations for downstream speech language models. However, ensuring accurate semantic modeling while maintaining high-fidelity reconstruction under ultra-low bitrate constraints remains challenging. We propose an entropy-guided group residual vector quantization (EG-GRVQ) for an ultra-low bitrate neural speech codec, which retains a semantic branch for linguistic information and incorporates an entropy-guided grouping strategy in the acoustic branch. Assuming that channel activations follow approximately Gaussian statistics, the variance of each channel can serve as a principled proxy for its information content. Based on this assumption, we partition the encoder output such that each group carries an equal share of the total information. This balanced allocation improves codebook efficiency and reduces redundancy. Trained on LibriTTS and VCTK, our model shows improvements in perceptual quality and intelligibility metrics under ultra-low bitrate conditions, with a focus on codec-level fidelity for communication-oriented scenarios.

Key facts. No citing papers in the map yet (data through 19 Oct 2026). Placed in Vocoders and speech synthesis, one of 11 directions in Speech and audio language models. Builds on 24 of its 30 references that are in the map. First posted on arXiv 2 Mar 2026.

Builds on

24 of 30 references.

Closest ancestors · 9

Same direction, newest first

  1. 2024Scaling Transformers for Low-Bitrate High-Quality Speech Coding29 Nov
  2. 2024MaskGCT: Zero-Shot Text-to-Speech with Masked Generative Codec Transformer1 Sep
  3. 2024WavTokenizer: an Efficient Acoustic Discrete Codec Tokenizer for Audio Language Modeling29 Aug
  4. 2023Vec-Tok Speech: speech vectorization and tokenization for neural speech generation11 Oct
  5. 2023SpeechTokenizer: Unified Speech Tokenizer for Speech Large Language Models31 Aug
  6. 2023AudioDec: An Open-source Streaming High-fidelity Neural Audio Codec26 May
  7. 2023HiFi-Codec: Group-residual Vector quantization for High Fidelity Audio Codec4 May
  8. 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers5 Jan
  9. 2021SoundStream: An End-to-End Neural Audio Codec7 Jul

Other references · 15

  1. 2017Neural Discrete Representation Learningcited by 4,843
  2. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 4,385
  3. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,762
  4. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,639
  5. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 2,135
  6. 2022High Fidelity Neural Audio Compressioncited by 917
  7. 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 864
  8. 2022AudioLM: a Language Modeling Approach to Audio Generationcited by 736
  9. 2024Moshi: a speech-text foundation model for real-time dialoguecited by 711
  10. 2018SDR - half-baked or well done?cited by 626
  11. 2023SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilitiescited by 618
  12. 2023SoundStorm: Efficient Parallel Audio Generationcited by 154
3 more
  1. 2024SemantiCodec: An Ultra Low Bitrate Semantic Audio Codec for General Soundcited by 84
  2. 2023Towards Universal Speech Discrete Tokens: A Case Study for ASR and TTScited by 44
  3. 2020Source-Aware Neural Speech Coding for Noisy Speech Compressioncited by 12

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025PURE Codec: Progressive Unfolding of Residual Entropy for Speech Codec Learning
  2. 2026An Ultra-Low-Bitrate Neural Speech Codec with Plain-to-Pseudo Synergistic Vector Quantization
  3. 2024ERVQ: Enhanced Residual Vector Quantization with Intra-and-Inter-Codebook Optimization for Neural Audio Codecs
  4. 2024ESC: Efficient Speech Coding with Cross-Scale Residual Vector Quantized Transformers
  5. 2026GS-Codec: A Gaussian-Splatting Bottleneck for Neural Audio Coding
  6. 2025U-Codec: Ultra Low Frame-rate Neural Speech Codec for Fast High-fidelity Speech Generation
  7. 2025Finite Scalar Quantization Enables Redundant and Transmission-Robust Neural Audio Compression at Low Bit-rates
  8. 2024Optimizing Neural Speech Codec for Low-Bitrate Compression via Multi-Scale Encoding