mlsanityData to 19 Oct 2026

Region-Aware Masking for Accent-Robust Cross-Lingual Text-to-Speech

arXiv 2610.07524 ↗First posted 5 Oct 2026eess.AScited by 0

On the map: Speech and audio language models → Vocoders and speech synthesis also fits: Speech translation

Abstract (arXiv)

Accent leakage remains a critical challenge in cross-lingual zero-shot text-to-speech (TTS), where models inadvertently transfer both the speaker's timbre and source-language accent into the target-language output. In mask-reconstruction TTS this is amplified by a train--inference mismatch: training reconstructs from same-language context, while inference conditions on cross-lingual context. We close this gap by concatenating utterances from two languages and placing the reconstruction mask relative to the language boundary, so training presents the cross-lingual prompting pattern the model encounters at inference. This requires no language IDs, accent labels, parallel same-speaker recordings, or architectural changes---only the mask geometry differs. In a listening study with native speakers, region-aware masking reduces perceived accent from 4.3 to 0.5 on a 0-5 scale against the unadapted baseline, and by 18-51% in seen regimes and 35-55% on unseen source languages against bilingual adaptation alone, with no measurable loss in intelligibility or naturalness. Comparing mask geometries shows that placement, not the amount masked, governs the balance between accent suppression and speaker preservation: masks confined to the reconstruction region suppress accent most but retain the reference speaker least reliably, whereas two-region masking achieves both. Mask geometry is thus a simple, effective lever for accent robustness.

Key facts. No citing papers in the map yet (data through 19 Oct 2026). Placed in Vocoders and speech synthesis, one of 11 directions in Speech and audio language models. Builds on 27 of its 29 references that are in the map. First posted on arXiv 5 Oct 2026.

Builds on

27 of 29 references.

Closest ancestors · 16

Same direction, newest first

  1. 2026CrossAccent-TTS: Cross-Lingual Accent-Intensity Controllable Text-to-Speech via Disentangled Speaker and Accent Representations24 Jun
  2. 2026X-Voice: Enabling Everyone to Speak 30 Languages via Zero-Shot Cross-Lingual Voice Cloning7 May
  3. 2026Activation Steering for Accent-Neutralized Zero-Shot Text-To-Speech6 Mar
  4. 2026PFluxTTS: Hybrid Flow-Matching TTS with Robust Cross-Lingual Voice Cloning and Inference-Time Model Fusion4 Feb
  5. 2025Cross-Lingual F5-TTS: Towards Language-Agnostic Voice Cloning and Speech Synthesis18 Sep
  6. 2024CrossSpeech++: Cross-lingual Speech Synthesis with Decoupled Language and Speaker Generation28 Dec
  7. 2024F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matching9 Oct
  8. 2024AccentBox: Towards High-Fidelity Zero-Shot Accent Generation13 Sep
  9. 2024E2 TTS: Embarrassingly Easy Fully Non-Autoregressive Zero-Shot TTS26 Jun
  10. 2024XTTS: a Massively Multilingual Zero-Shot Text-to-Speech Model7 Jun
  11. 2023DSE-TTS: Dual Speaker Embedding for Cross-Lingual Text-to-Speech25 Jun
  12. 2023Voicebox: Text-Guided Multilingual Universal Speech Generation at Scale23 Jun
4 more
  1. 2023Speak Foreign Languages with Your Own Voice: Cross-Lingual Neural Codec Language Modeling7 Mar
  2. 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers5 Jan
  3. 2022ERNIE-SAT: Speech and Text Joint Pretraining for Cross-Lingual Multi-Speaker Text-to-Speech7 Nov
  4. 2021YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyone4 Dec

Other references · 11

  1. 2022Flow Matching for Generative Modelingcited by 5,905
  2. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,661
  3. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 2,135
  4. 2020ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verificationcited by 980
  5. 2023Improving and generalizing flow-based generative models with minibatch optimal transportcited by 908
  6. 2022UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022cited by 552
  7. 2022FLEURS: Few-shot Learning Evaluation of Universal Representations of Speechcited by 539
  8. 2019Learning to Speak Fluently in a Foreign Language: Multilingual Speech Synthesis and Cross-Language Voice Cloningcited by 85
  9. 2022SpeechPainter: Text-conditioned Speech Inpaintingcited by 19
  10. 2025Accent Normalization Using Self-Supervised Discrete Tokens with Non-Parallel Datacited by 6
  11. 2022Data-augmented cross-lingual synthesis in a teacher-student frameworkcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Accent Analogy Guidance: More Speaker Similarity at Equal Accent in Cross-Lingual Voice Cloning
  2. 2026DEFINE: Exemplar-Guided Accent Control for Zero-Shot TTS
  3. 2026Phrase-Localized Language-Contrastive Guidance: Training-Free Localized Accent Control for Code-Switching Text-to-Speech
  4. 2025CLARITY: Contextual Linguistic Adaptation and Accent Retrieval for Dual-Bias Mitigation in Text-to-Speech Generation
  5. 2025Unseen Speaker and Language Adaptation for Lightweight Text-To-Speech with Adapters
  6. 2026Joycent: Multi-Accent TTS via Disentangled Accent Modeling and Layer-Specific Conditioning
  7. 2026Activation Steering for Accent Adaptation in Large Audio Language Models
  8. 2025Languages in Whisper-Style Speech Encoders Align Both Phonetically and Semantically