# Environment Aware Text-to-Speech Synthesis

arXiv: https://arxiv.org/abs/2110.03887 · HTML: https://mlsanity.com/paper/2110.03887/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Oct 2021.

## Built on by (most cited first, 4 of 4)

- [Self-Supervised Visual Acoustic Matching](https://mlsanity.com/paper/2307.15064/) (2023)
- [Multi-Source Spatial Knowledge Understanding for Immersive Visual Text-to-Speech](https://mlsanity.com/paper/2410.14101/) (2024)
- [VividVoice: A Unified Framework for Scene-Aware Visually-Driven Speech Synthesis](https://mlsanity.com/paper/2602.02591/) (2026)
- [Towards Real-world Environment-aware Zero-shot Text-to-speech Synthesis via Disentangled Audio Infilling](https://mlsanity.com/paper/2608.03011/) (2026)

## Abstract (arXiv)

This study aims at designing an environment-aware text-to-speech (TTS) system that can generate speech to suit specific acoustic environments. It is also motivated by the desire to leverage massive data of speech audio from heterogeneous sources in TTS system development. The key idea is to model the acoustic environment in speech audio as a factor of data variability and incorporate it as a condition in the process of neural network based speech synthesis. Two embedding extractors are trained with two purposely constructed datasets for characterization and disentanglement of speaker and environment factors in speech. A neural network model is trained to generate speech from extracted speaker and environment embeddings. Objective and subjective evaluation results demonstrate that the proposed TTS system is able to effectively disentangle speaker and environment factors and synthesize speech audio that carries designated speaker characteristics and environment attribute. Audio samples are available online for demonstration https://daxintan-cuhk.github.io/Environment-Aware-TTS/ .