Soft Disentanglement in Frequency Bands for Neural Audio Codecs
On the map: Speech and audio language models → Vocoders and speech synthesis
Abstract (arXiv)
In neural-based audio feature extraction, ensuring that representations capture disentangled information is crucial for model interpretability. However, existing disentanglement methods often rely on assumptions that are highly dependent on data characteristics or specific tasks. In this work, we introduce a generalizable approach for learning disentangled features within a neural architecture. Our method applies spectral decomposition to time-domain signals, followed by a multi-branch audio codec that operates on the decomposed components. Empirical evaluations demonstrate that our approach achieves better reconstruction and perceptual performance compared to a state-of-the-art baseline while also offering potential advantages for inpainting tasks.
Key facts. Cited by 2 papers in the map (data through 19 Oct 2026). Placed in Vocoders and speech synthesis, one of 11 directions in Speech and audio language models. Builds on 19 of its 30 references that are in the map. First posted on arXiv 4 Oct 2025.
Builds on
19 of 30 references.
Closest ancestors · 5
Same direction, newest first
- 2024Learning Source Disentanglement in Neural Audio Codec17 Sep
- 2024NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Models5 Mar
- 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers5 Jan
- 2022Disentangling speech from surroundings with neural embeddings29 Mar
- 2021SoundStream: An End-to-End Neural Audio Codec7 Jul
Other references · 14
- 2017Neural Discrete Representation Learningcited by 4,843
- 2022High Fidelity Neural Audio Compressioncited by 917
- 2018SDR - half-baked or well done?cited by 626
- 2023High-Fidelity Audio Compression with Improved RVQGANcited by 619
- 2021Speech Resynthesis from Discrete Disentangled Self-Supervised Representationscited by 287
- 2020ViSQOL v3: An Open Source Production Ready Objective Speech and Audio Metriccited by 120
- 2022SQ-VAE: Variational Bayes on Discrete Representation with Self-annealed Stochastic Quantizationcited by 75
- 2020Hierarchical Quantized Autoencoderscited by 58
- 2023Disentanglement via Latent Quantizationcited by 43
- 2023HQ-VAE: Hierarchical Discrete Representation Learning with Variational Bayescited by 29
- 2021Self-Supervised VQ-VAE for One-Shot Music Style Transfercited by 17
- 2021Hierarchical disentangled representation learning for singing voice conversioncited by 13
2 more
- 2024Gull: A Generative Multifunctional Audio Codeccited by 10
- 2021An Exploratory Study on Perceptual Spaces of the Singing Voicecited by 4
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2025D'esentrelacement Fr'equentiel Doux pour les Codecs Audio Neuronaux
- 2025DeCodec: Rethinking Audio Codecs as Universal Disentangled Representation Learners
- 2022Disentangled Feature Learning for Real-Time Neural Speech Coding
- 2025Exploring Disentangled Neural Speech Codecs from Self-Supervised Representations
- 2022Towards Disentangled Speech Representations
- 2025BSCodec: A Band-Split Neural Codec for High-Quality Universal Audio Reconstruction
- 2026Feature Encoding in VAE-based Audio Decoders: Effects of Input, Depth and Distribution
- 2025ComplexDec: A Domain-robust High-fidelity Neural Audio Codec with Complex Spectrum Modeling