Rethinking Discrete Speech Representation Tokens for Accent Generation
Not on the map. Why?
Abstract (arXiv)
Discrete Speech Representation Tokens (DSRTs) have become a foundational component in speech generation. While prior work has extensively studied phonetic and speaker information in DSRTs, how accent information is encoded in DSRTs remains largely unexplored. In this paper, we present the first systematic investigation of accent information in DSRTs. We propose a unified evaluation framework that measures both accessibility of accent information via a novel Accent ABX task and recoverability via cross-accent Voice Conversion (VC) resynthesis. Using this framework, we analyse DSRTs derived from several widely used speech representations. Our results reveal that: (1) choice of layers has the most significant impact on retaining accent information, (2) accent information is substantially reduced by ASR supervision; (3) naive codebook size reduction cannot effectively disentangle accent from phonetic and speaker information.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 29 of its 39 references that are in the map. First posted on arXiv 27 Jan 2026.
Builds on
29 of 39 references.
References · 29
- 2017Neural Discrete Representation Learningcited by 4,650
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
- 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
- 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
- 2021SoundStream: An End-to-End Neural Audio Codeccited by 756
- 2022AudioLM: a Language Modeling Approach to Audio Generationcited by 611
- 2021XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scalecited by 589
- 2024Moshi: a speech-text foundation model for real-time dialoguecited by 536
- 2023Finite Scalar Quantization: VQ-VAE Made Simplecited by 425
- 2021W2v-BERT: Combining Contrastive Learning and Masked Language Modeling for Self-Supervised Speech Pre-Trainingcited by 340
- 2024CosyVoice: A Scalable Multilingual Zero-shot Text-to-speech Synthesizer based on Supervised Semantic Tokenscited by 323
- 2021Layer-wise Analysis of a Self-supervised Speech Representation Modelcited by 293
17 more
- 2021Speech Resynthesis from Discrete Disentangled Self-Supervised Representationscited by 243
- 2023Speak, Read and Prompt: High-Fidelity Text-to-Speech with Minimal Supervisioncited by 167
- 2024MaskGCT: Zero-Shot Text-to-Speech with Masked Generative Codec Transformercited by 159
- 2021Direct speech-to-speech translation with discrete unitscited by 148
- 2022Generative Spoken Dialogue Language Modelingcited by 125
- 2024Recent Advances in Speech Language Models: A Surveycited by 85
- 2024Self-Supervised Speech Representations are More Phonetic than Semanticcited by 70
- 2025Recent Advances in Discrete Speech Tokens: A Reviewcited by 42
- 2023HierSpeech++: Bridging the Gap between Semantic and Acoustic Representation of Speech by Hierarchical Variational Inference for Zero-shot Speech Synthesiscited by 35
- 2025Vevo: Controllable Zero-Shot Voice Imitation with Self-Supervised Disentanglementcited by 34
- 2023RepCodec: A Speech Representation Codec for Speech Tokenizationcited by 27
- 2023CommonAccent: Exploring Large Acoustic Pretrained Models for Accent Classification Based on Common Voicecited by 19
- 2024High-Fidelity Neural Phonetic Posteriorgramscited by 9
- 2024Estimating the Completeness of Discrete Speech Unitscited by 9
- 2024AccentBox: Towards High-Fidelity Zero-Shot Accent Generationcited by 9
- 2025Pairwise Evaluation of Accent Similarity in Speech Synthesiscited by 4
- 2022Is the Language Familiarity Effect gradual? A computational modelling approachcited by 2
Built on by
Citing papers per year: 2026: 2.
2 papers.