# Frame-Wise Breath Detection with Self-Training: An Exploration of Enhancing Breath Naturalness in Text-to-Speech

arXiv: https://arxiv.org/abs/2402.00288 · HTML: https://mlsanity.com/paper/2402.00288/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Feb 2024.

## Built on by (most cited first, 2 of 2)

- [Speculative End-Turn Detector for Efficient Speech Chatbot Assistant](https://mlsanity.com/paper/2503.23439/) (2025)
- [An Initial Study of Physiological Markers for Deepfake Speech Detection](https://mlsanity.com/paper/2610.18934/) (2026)

## Abstract (arXiv)

Developing Text-to-Speech (TTS) systems that can synthesize natural breath is essential for human-like voice agents but requires extensive manual annotation of breath positions in training data. To this end, we propose a self-training method for training a breath detection model that can automatically detect breath positions in speech. Our method trains the model using a large speech corpus and involves: 1) annotation of limited breath sounds utilizing a rule-based approach, and 2) iterative augmentation of these annotations through pseudo-labeling based on the model's predictions. Our detection model employs Conformer blocks with down-/up-sampling layers, enabling accurate frame-wise breath detection. We investigate its effectiveness in multi-speaker TTS using text transcripts with detected breath marks. The results indicate that using our proposed model for breath detection and breath mark insertion synthesizes breath-contained speech more naturally than a baseline model.