# Machine Translation Verbosity Control for Automatic Dubbing

arXiv: https://arxiv.org/abs/2110.03847 · HTML: https://mlsanity.com/paper/2110.03847/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 32 references that are in the map.
- First posted on arXiv 8 Oct 2021.

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation](https://mlsanity.com/paper/1609.08144/) (2016)
- [Sequence-to-Sequence Models Can Directly Translate Foreign Speech](https://mlsanity.com/paper/1703.08581/) (2017)
- [Controllable Abstractive Summarization](https://mlsanity.com/paper/1711.05217/) (2017)
- [Speech Translation and the End-to-End Promise: Taking Stock of Where We Are](https://mlsanity.com/paper/2004.06358/) (2020)
- [Controlling Output Length in Neural Encoder-Decoders](https://mlsanity.com/paper/1609.09552/) (2016)
- [Positional Encoding to Control Output Sequence Length](https://mlsanity.com/paper/1904.07418/) (2019)
- [Controlling the Output Length of Neural Machine Translation](https://mlsanity.com/paper/1910.10408/) (2019)
- [Controlling Text Complexity in Neural Machine Translation](https://mlsanity.com/paper/1911.00835/) (2019)
- [Extreme Adaptation for Personalized Neural Machine Translation](https://mlsanity.com/paper/1805.01817/) (2018)
- [From Speech-to-Speech Translation to Automatic Dubbing](https://mlsanity.com/paper/2001.06785/) (2020)
- [Effect of data reduction on sequence-to-sequence neural TTS](https://mlsanity.com/paper/1811.06315/) (2018)
- [Prosodic Phrase Alignment for Machine Dubbing](https://mlsanity.com/paper/1908.07226/) (2019)
- [In Other News: A Bi-style Text-to-speech Model for Synthesizing Newscaster Voice with Limited Data](https://mlsanity.com/paper/1904.02790/) (2019)
- [Machine Translation with Unsupervised Length-Constraints](https://mlsanity.com/paper/2004.03176/) (2020)

## Built on by (most cited first, 18 of 18)

- [CoCoA-MT: A Dataset and Benchmark for Contrastive Controlled MT with Application to Formality](https://mlsanity.com/paper/2205.04022/) (2022)
- [VideoDubber: Machine Translation with Speech-Aware Length Control for Video Dubbing](https://mlsanity.com/paper/2211.16934/) (2022)
- [Isometric MT: Neural Machine Translation for Automatic Dubbing](https://mlsanity.com/paper/2112.08682/) (2021)
- [Dubbing in Practice: A Large Scale Study of Human Localization With Insights for Automatic Dubbing](https://mlsanity.com/paper/2212.12137/) (2022)
- [Isochrony-Aware Neural Machine Translation for Automatic Dubbing](https://mlsanity.com/paper/2112.08548/) (2021)
- [Direct Speech Translation for Automatic Subtitling](https://mlsanity.com/paper/2209.13192/) (2022)
- [ANIM-400K: A Large-Scale Dataset for Automated End-To-End Dubbing of Video](https://mlsanity.com/paper/2401.05314/) (2024)
- [Jointly Optimizing Translations and Speech Timing to Improve Isochrony in Automatic Dubbing](https://mlsanity.com/paper/2302.12979/) (2023)
- [Isometric Neural Machine Translation using Phoneme Count Ratio Reward-based Reinforcement Learning](https://mlsanity.com/paper/2403.15469/) (2024)
- [Controlling Extra-Textual Attributes about Dialogue Participants -- A Case Study of English-to-Polish Neural Machine Translation](https://mlsanity.com/paper/2205.04747/) (2022)
- [AppTek's Submission to the IWSLT 2022 Isometric Spoken Language Translation Task](https://mlsanity.com/paper/2205.05807/) (2022)
- [Improving Isochronous Machine Translation with Target Factors and Auxiliary Counters](https://mlsanity.com/paper/2305.13204/) (2023)
- [Prompting LLMs: Length Control for Isometric Machine Translation](https://mlsanity.com/paper/2506.04855/) (2025)
- [IsoChronoMeter: A simple and effective isochronic translation evaluation metric](https://mlsanity.com/paper/2410.11127/) (2024)
- [SparQLe: Speech Queries to Text Translation Through LLMs](https://mlsanity.com/paper/2502.09284/) (2025)
- [Length Aware Speech Translation for Video Dubbing](https://mlsanity.com/paper/2506.00740/) (2025)
- [Evaluating the Efficacy of Length-Controllable Machine Translation](https://mlsanity.com/paper/2305.02300/) (2023)
- [PS-TTS: Phonetic Synchronization in Text-to-Speech for Achieving Natural Automated Dubbing](https://mlsanity.com/paper/2604.09111/) (2026)

## Abstract (arXiv)

Automatic dubbing aims at seamlessly replacing the speech in a video document with synthetic speech in a different language. The task implies many challenges, one of which is generating translations that not only convey the original content, but also match the duration of the corresponding utterances. In this paper, we focus on the problem of controlling the verbosity of machine translation output, so that subsequent steps of our automatic dubbing pipeline can generate dubs of better quality. We propose new methods to control the verbosity of MT output and compare them against the state of the art with both intrinsic and extrinsic evaluations. For our experiments we use a public data set to dub English speeches into French, Italian, German and Spanish. Finally, we report extensive subjective tests that measure the impact of MT verbosity control on the final quality of dubbed video clips.