mlsanityData to 7 Oct 2026

Zero-Shot Lombard Speech Synthesis with Controllable Style Embeddings

arXiv 2601.12966 ↗First posted 19 Jan 2026cs.SD cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

The Lombard effect plays a key role in natural communication, particularly in noisy environments or when addressing hearing-impaired listeners. We present a controllable text-to-speech (TTS) system capable of synthesizing Lombard-like speech in a zero-shot manner without requiring Lombard-specific training data. Our approach extends F5-TTS with a learned style embedding representation and analyzes the resulting latent space using principal component analysis (PCA) to identify directions associated with Lombard-related attributes. By manipulating these directions, we obtain interpretable control over vocal effort and articulation and generate speech at different Lombard levels. Experimental results show that the proposed method preserves speaker identity and naturalness, improves intelligibility under noisy conditions, and generalizes to previously unseen speakers. These findings demonstrate that style-embedding manipulation provides an effective and scalable framework for controllable zero-shot Lombard speech synthesis.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 8 of its 37 references that are in the map. First posted on arXiv 19 Jan 2026.

Builds on

8 of 37 references.

References · 8

  1. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,286
  2. 2020MLS: A Large-Scale Multilingual Dataset for Speech Researchcited by 467
  3. 2022UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022cited by 381
  4. 2024F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matchingcited by 353
  5. 2024Emilia: An Extensive, Multilingual, and Diverse Speech Dataset for Large-Scale Speech Generationcited by 200
  6. 2020Enhancing Speech Intelligibility in Text-To-Speech Synthesis using Speaking Style Conversioncited by 6
  7. 2021Whispered and Lombard Neural Speech Synthesiscited by 6
  8. 2022Vocal effort modeling in neural TTS for improving the intelligibility of synthetic speech in noisecited by 3

Built on by

Not cited yet.