# Multi-Task Learning for Front-End Text Processing in TTS

arXiv: https://arxiv.org/abs/2401.06321 · HTML: https://mlsanity.com/paper/2401.06321/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 35 references that are in the map.
- First posted on arXiv 12 Jan 2024.

## Builds on (most cited first, 12 of 12)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [Multi-Task Deep Neural Networks for Natural Language Understanding](https://mlsanity.com/paper/1901.11504/) (2019)
- [Multi-task Sequence to Sequence Learning](https://mlsanity.com/paper/1511.06114/) (2015)
- [RNN Approaches to Text Normalization: A Challenge](https://mlsanity.com/paper/1611.00068/) (2016)
- [Morphosyntactic Tagging with a Meta-BiLSTM Model over Context Sensitive Token Encodings](https://mlsanity.com/paper/1805.08237/) (2018)
- [Multi-Task Learning for Sequence Tagging: An Empirical Study](https://mlsanity.com/paper/1808.04151/) (2018)
- [Scalable Multilingual Frontend for TTS](https://mlsanity.com/paper/2004.04934/) (2020)
- [Proteno: Text Normalization with Limited Data for Fast Deployment in Text to Speech Systems](https://mlsanity.com/paper/2104.07777/) (2021)
- [Transformer-based Models of Text Normalization for Speech Applications](https://mlsanity.com/paper/2202.00153/) (2022)
- [A unified front-end framework for English text-to-speech synthesis](https://mlsanity.com/paper/2305.10666/) (2023)

## Built on by (most cited first, 1 of 1)

- [Acquiring Pronunciation Knowledge from Transcribed Speech Audio via Multi-task Learning](https://mlsanity.com/paper/2409.09891/) (2024)

## Abstract (arXiv)

We propose a multi-task learning (MTL) model for jointly performing three tasks that are commonly solved in a text-to-speech (TTS) front-end: text normalization (TN), part-of-speech (POS) tagging, and homograph disambiguation (HD). Our framework utilizes a tree-like structure with a trunk that learns shared representations, followed by separate task-specific heads. We further incorporate a pre-trained language model to utilize its built-in lexical and contextual knowledge, and study how to best use its embeddings so as to most effectively benefit our multi-task model. Through task-wise ablations, we show that our full model trained on all three tasks achieves the strongest overall performance compared to models trained on individual or sub-combinations of tasks, confirming the advantages of our MTL framework. Finally, we introduce a new HD dataset containing a balanced number of sentences in diverse contexts for a variety of homographs and their pronunciations. We demonstrate that incorporating this dataset into training significantly improves HD performance over only using a commonly used, but imbalanced, pre-existing dataset.