# Exploring Disentangled Neural Speech Codecs from Self-Supervised Representations

arXiv: https://arxiv.org/abs/2508.08399 · HTML: https://mlsanity.com/paper/2508.08399/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Aug 2025.

## Built on by (most cited first, 3 of 3)

- [DSpAST: Disentangled Representations for Spatial Audio Reasoning with Large Language Models](https://mlsanity.com/paper/2509.13927/) (2025)
- [Self-supervised Speech Comparison for L2 Phone, Rhythm, and Intonation Scoring](https://mlsanity.com/paper/2607.13721/) (2026)
- [NouveauVoice: Generating Novel Pseudo Speakers for Voice Anonymization](https://mlsanity.com/paper/2607.03985/) (2026)

## Abstract (arXiv)

Neural audio codecs (NACs), which use neural networks to generate compact audio representations, have garnered interest for their applicability to many downstream tasks -- especially quantized codecs due to their compatibility with large language models. However, unlike text, speech conveys not only linguistic content but also rich paralinguistic features. Encoding these elements in an entangled fashion may be suboptimal, as it limits flexibility. For instance, voice conversion (VC) aims to convert speaker characteristics while preserving the original linguistic content, which requires a disentangled representation. Inspired by VC methods utilizing k-means quantization with self-supervised features to disentangle phonetic information, we develop a discrete NAC capable of structured disentanglement. Experimental evaluations show that our approach achieves reconstruction performance on par with conventional NACs that do not explicitly perform disentanglement, while also matching the effectiveness of conventional VC techniques.