Two-Dimensional Quantization for Geometry-Aware Audio Coding
Not on the map. Why?
Abstract (arXiv)
Recent neural audio codecs have achieved impressive reconstruction quality, typically relying on quantization methods such as Residual Vector Quantization (RVQ), Vector Quantization (VQ) and Finite Scalar Quantization (FSQ). However, these quantization techniques limit the geometric structure of the latent space, make it harder to capture correlations between features leading to inefficiency in representation learning, codebook utilization and token rate. In this paper we introduce Two-Dimensional Quantization (Q2D2), a quantization scheme in which feature pairs are projected onto structured 2D grids, such as hexagonal, rhombic, or rectangular tiling and quantized to the nearest grid values, yielding an implicit codebook defined by the product of grid levels, with codebook sizes comparable to conventional methods. Despite its simple geometric formulation, Q2D2 improves audio compression efficiency, with low token rates and high codebook utilization while maintaining state of the art reconstruction quality. Specifically, Q2D2 achieves competitive to superior performance in various objective and subjective reconstruction metrics, across extensive experiments in speech, audio and music domains compared to state of the art models. Comprehensive ablation studies further confirm the effectiveness of our design choices.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 36 of its 71 references that are in the map. First posted on arXiv 1 Dec 2025.
Builds on
36 of 71 references.
References · 36
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2017Neural Discrete Representation Learningcited by 4,650
- 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
- 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,358
- 2021SoundStream: An End-to-End Neural Audio Codeccited by 756
- 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 737
- 2022data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Languagecited by 718
- 2022Autoregressive Image Generation using Residual Quantizationcited by 706
- 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 639
- 2020Jukebox: A Generative Model for Musiccited by 535
- 2023SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilitiescited by 518
- 2020MLS: A Large-Scale Multilingual Dataset for Speech Researchcited by 467
24 more
- 2023Finite Scalar Quantization: VQ-VAE Made Simplecited by 425
- 2023High-Fidelity Audio Compression with Improved RVQGANcited by 423
- 2020FSD50K: An Open Dataset of Human-Labeled Sound Eventscited by 339
- 2024Seed-TTS: A Family of High-Quality Versatile Speech Generation Modelscited by 293
- 2020SLURP: A Spoken Language Understanding Resource Packagecited by 197
- 2023Speak, Read and Prompt: High-Fidelity Text-to-Speech with Minimal Supervisioncited by 167
- 2023Vocos: Closing the gap between time-domain and Fourier-based neural vocoders for high-quality audio synthesiscited by 150
- 2023HiFi-Codec: Group-residual Vector quantization for High Fidelity Audio Codeccited by 95
- 2023Straightening Out the Straight-Through Estimator: Overcoming Optimization Challenges in Vector Quantized Networkscited by 80
- 2024Addressing Representation Collapse in Vector Quantized Models with One Linear Layercited by 77
- 2018AudioMNIST: Exploring Explainable Artificial Intelligence for Audio Analysis on a Simple Benchmarkcited by 72
- 2024Codec Does Matter: Exploring the Semantic Shortcoming of Codec for Audio Language Modelcited by 70
- 2022SQ-VAE: Variational Bayes on Discrete Representation with Self-annealed Stochastic Quantizationcited by 68
- 2020Robust Training of Vector Quantized Bottleneck Modelscited by 53
- 2023Mega-TTS: Zero-Shot Text-to-Speech at Scale with Intrinsic Inductive Biascited by 51
- 2024Scaling Transformers for Low-Bitrate High-Quality Speech Codingcited by 48
- 2023AudioDec: An Open-source Streaming High-fidelity Neural Audio Codeccited by 47
- 2024BigCodec: Pushing the Limits of Low-Bitrate Neural Speech Codeccited by 45
- 2023FunCodec: A Fundamental, Reproducible and Integrable Open-source Toolkit for Neural Speech Codeccited by 44
- 2024SemantiCodec: An Ultra Low Bitrate Semantic Audio Codec for General Soundcited by 42
- 2023RepCodec: A Speech Representation Codec for Speech Tokenizationcited by 27
- 2024Single-Codec: Single-Codebook Speech Codec towards High-Performance Speech Generationcited by 23
- 2023Fewer-token Neural Speech Codec with Time-invariant Codescited by 21
- 2024PSCodec: A Series of High-Fidelity Low-bitrate Neural Speech Codecs Leveraging Prompt Encoderscited by 3
Built on by
Not cited yet.